Page 1 of 1

Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: February 24th, 2020, 5:37 pm
by sickle
e.g.

ROM @$1f800000 -> $1F020690
EXE @$1f020690 -> $something

The .exe works fine normally (as in, with a header in the $8000 range) and is identical whether built with /P (binary output) or if you strip the $800 header out.

Startup on the left (essentially replicating GCC's crt0)
Linker on the right - with the data and text groups defined separately:

Note: 'org' completely ignored if you manually define every section
E.g. you can org any random value and the resultant .exe is identical

Image


It gets decently far into my C code:

-> Boot @ 1F000000
-> Loads address of the incbin'd .exe and jr t0's there
-> Loads the .crt0 (.global in IDA)
-> makes it into main() and over a few variable initalisations
-> hangs on ResetGraph(0)

Adding the asm chunk to copy ROM sections into RAM didn't seem to help at all.
Note: i'm aware it's copying readonly chunks, but it was easier to copy one massive chunk for testing

Can this even be done in PSYQ?
E.g. we have the .bss and .sbss sections - .sbss seems to work

Adding some dummy bytes for .sdata for example fixed the
la $gp,0
issue, but that's as far as I've got.

Doing something like this
(or the other way around - leaving bss undefined and defining .text)
Image

Throws this error
Image


Any way to debug this? Just a bit lost right now.
Any tips would be appreciated

Cheers =)

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: February 25th, 2020, 9:20 am
by sickle
Might as well keep a running log of this for that one in case anyone ever runs into the same issue:

First thing, you can org things like this:

Code: Select all

		org	$1F020710
text	group 	obj($1F020710)
	
		section	.text,text

		section	.ctor,text
		section	.dtors,text



		org	$80010000
othertext	group	obj($80010000)
		
		section	.rdata,othertext	

		section	.data,othertext
		section	.sdata,othertext
		
		;org	$80010000
bss		group	bss
		
		section	.sbss,bss		; global vars
		section	.bss,bss		; global vars


For some reason, putting the mutable and immutable variables nearer eachother (via a third group: otherText) has fixed some of the linker/compilation issues. bss can now be correctly grouped as bss.

There are also some oddities based on which order you put the org statements, which ones are omited, etc.
Some configurations produce outright broken builds (e.g. blatantly wrong jump instructions leading off to nowhere)

- Works fine if I add fastboot, relocate the stack, etc.
- The linker does properly produce code that works nicely in the $1F000000 region, while reading vars from $80000000
- Temporary variable storage for InitHeap (storing copies of A0, A1, etc) does work
( can also verify via IDA and basic C code that it is *writing* to RAM, need to check reading)
- Still hangs on ResetGraph( 0 )

Few things I need to verify next:

- build with loads of these to quickly verify section sizes in the debugger
la t0,sect( .whateverSection )
la t0,sectEnd( whateverSection )

- few simple checks - actual vs expected vars for each section

- rebuild the whole thing as a standalone .exe org'd to $80010000 (e.g code's there), but which relocates its variables through startup.s to $80020000. At least then it's debuggable, lol.

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 1st, 2020, 5:46 am
by sickle
After a bit of debugging, I got it working :D
Will leave some details since there's not much info on this.

So the secret ingredient was disabling GP register optimisations.

E.g. from this:

Code: Select all

ccpsx -c -DLINKED_STARTUP -O0 -ounirommain.obj main.c
To this:

Code: Select all

ccpsx -c -DLINKED_STARTUP -O0 -G0 -ounirommain.obj main.c
Without that, GP is pointed at the start of your vars, then offset by 2 bytes for reads/writes
E.g. your first and second ints might be loaded in like:

Code: Select all

li t0,$0(gp)
li t1,$4(gp)
... etc
Trading speed for limiting us to a range of only 2 bytes, since GP will generally not change.

Second thing was not being so hung up on bss.
E.g. it's declared as
bss group bss

The first 'bss' being your label and the second 'bss' being some kinda internal keyword that I can't find much info on.
Before settling on the current org/group layout things were getting crashy without it. It does seem to work however if you just omit it and group your .bss/ssbss with something else.


Current linker setup:

Code: Select all

			;   $1F000000 = Rom Start
			; + $00020000 = End of Caetla *
			; + $00000700 = End of Unirom Bootloader
			; + $00000010 = 10 random btes
			; = $1F020710
			; * Caetla's entry point has been patched to
			;   a static entry in the bootloader at $1F020000

			; Machine code at $1F00**** in ROM
			; Vars at $8001**** in main RAM

			org		$1F020710

text		group 	obj($1F020710)			
othertext	group	obj($80020000)
bss			group	obj($80030000)
			
			section	.text,text
			;section	.ctor,text
			;section	.dtors,text
			
			section	.data,othertext
			section	.sdata,othertext
			section	.rdata,othertext
			
			section	.sbss,bss
			section	.bss,bss


			; original layout
			;section	.rdata,text		; readonly data
			;section	.text,text		; 
			;section	.sdata,text		; vars
			;section	.sbss,bss		; global vars
			;section	.bss,bss		; global vars

; From SN, thought it would be useful:
; Use can override these settings by declaring suitable values
; for these variables in his C code module along with main().
; e.g.
;_stacksize	equ	$00002000	; 8K
;_ramsize	equ	$00200000	; 2MB
_testboot	equ	$BFC00000	

; From SN docs:
; If user does not specify override values for these variables
; by defining them as unsigned int then the default values will
; be loaded from the SNDEF module in LIBSN.LIB

		include		startup.o
		include		unirommain.obj    
		include		asm_helperfunctions.obj
		include		asm_CDWrapper.obj


; let's not include libsn unless we have to
		;inclib   "c:\psyq\lib\libsn.lib"
		inclib	"c:\psyq\lib\libapi.lib"
		inclib   "c:\psyq\lib\libapi.lib"
		inclib   "c:\psyq\lib\libgpu.lib"
		inclib   "c:\psyq\lib\libetc.lib"
		inclib   "c:\psyq\lib\libcd.lib"
		inclib   "c:\psyq\lib\libpress.lib"
		inclib   "c:\psyq\lib\libc.lib"
		inclib   "c:\psyq\lib\libsio.lib"
		inclib   "c:\psyq\lib\libgpu.lib"
		inclib   "c:\psyq\lib\libgs.lib"
		inclib   "c:\psyq\lib\libgte.lib"


		;regs	pc = _testboot
		regs	pc=__SN_ENTRY_POINT	; entry point


I'll get the code up on github if it remains pretty stable though development.

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 1st, 2020, 6:12 am
by rama3
Thanks for sharing your findings, even if I can't make sense of it :)

Is this to allow embedding more psx executables?
I could really use a cheat cart with "PSX ROMid 1.7.0" on it :p
(This tool:

No idea if it can be made to fit though. Maybe on the 512k Xploders.

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 1st, 2020, 11:15 pm
by sickle
rama3 wrote: March 1st, 2020, 6:12 am can't make sense of it
Oh man, it's not nearly as complex as it sounds.
Basically just tweaking the compiler so the program part runs from ROM but the variables are in main memory.
E.g. the code doesn't change, but the variables have to.

Normally not a problem 'cause you can just extract an .exe from rom and run it normally..
But that does take up some memory that you then can't use for other stuff.
So I've just been trying to run *most* of it from ROM and use main mem for some variables.
Wasnt' a problem before 'cause old version was all in assembly, but it was taking too long to get shit done.
rama3 wrote: March 1st, 2020, 6:12 am ...No idea if it can be made to fit though. Maybe on the 512k Xploders.
Should be fine, yeah.
The current version of UniROM allows you to embedd random .exes like that if you want.
I think the batch files is "build_unirom_plugin"

- copy the .exe to the end of the rom
- burn it on a disk or flash it over
- install using xflash
- you can now use triangle (i think) to launch it

If you don't use caetla, it should be able to fit on most carts.

Got a copy of the exe?

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 2nd, 2020, 11:32 am
by rama3
Sure,
attached is the PSX.EXE, extracted with power iso :)

Its SYSTEM.CNF looks typical:

Code: Select all

BOOT = cdrom:\PSX.EXE;1
TCB = 4
EVENT = 16
STACK = 801FFF00
The file isn't even that big. I don't remember what the constraints are for plugins right now, but this might fit onto the current release even.

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 3rd, 2020, 2:10 am
by sickle
Cheers for the .exe!

I've added it to the rom for you:
This is a kinda pre-release testy version of Unirom7b, so it's not got all the features, but your bios tool is attached.
Hit square and wait a few seconds. Seems to work :)
unirom_caetla_plugin.rom
unirom_caetlaNTSC_plugin.rom
unirom_standalone_plugin.rom
J

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 3rd, 2020, 8:31 am
by rama3
Thanks, you're awesome! :)

Edit:
Works well it seems. I used the new nxflash with romfile.exe to create a flasher CD.
Edit2:
But neither of the 2 CD boot options work (you probably know this :p).

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 4th, 2020, 9:46 am
by sickle
rama3 wrote: March 3rd, 2020, 8:31 am But neither of the 2 CD boot options work (you probably know this :p).
Lmao, now you mention it... I do seem to have removed the working one :lol:
Have a look at UniROM 6 if you need slightly more stable booting.
(I think it's also smaller 'cause stuff's decompressed from rom into ram)

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 4th, 2020, 9:03 pm
by rama3
Oh, I don't mind :)
The build gives me quick access to CD controller commands, without even a CD inserted.
It's all I need for development :p

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 4th, 2020, 9:54 pm
by sickle
Hah, what you doing? :D I'm so intrigued?

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 5th, 2020, 3:38 am
by rama3
Call it an expansion to my PsNee mod ;)

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 5th, 2020, 11:18 am
by sickle
Ahh sweet, looking forward to it ^^

Re: Trying to run a headerless PSYQ .exe from ROM in PSYQ

Posted: March 11th, 2020, 10:52 am
by sickle
More notes for future people.


There are some solid reasons to *not* use this approach
(.EXE in memory, variables in RAM)


1:
You'll need to know your entry point exactly.
The ROM area doesn't boot from the start, but rather a pointer to somewhere after the header, and you need to work that out.

Simple enough to put the .text section first in the .exe and jump there without a header... but then IDA doesn't like it.
Either way it's a faff having to update your org and .text obj() each time and you have to constantly watch your space, or pre-allocate space to a known boundary.

vs

just extract your .exe...


2:
Your linker file will need 3 section declarations:

.bss and .sbss *need* to be part of a 'bss' group
whether that's

bss group bss
or
bss group obj($80100000)



But obviously you need the .text where it's org'd (on the ROM), and somewhere for the other variables...
I went with "otherText".
That's fine and all, but it's a bit of a guess as to the memory spacing.
E.g. if you put 100kb between otherText and bss, your file size will increase by whatever the difference in the 2 was

vs

just extract your *much smaller* .exe (i'm getting 70kb vs 100)


3:

Can't compress it on the ROM

vs

Decompress into mem for almost no time hit.

4:

IDA pro hates it when you rearrange sections *or* use a nonstandard startup assembly.

vs

IDA works fine.

5:

Flashing the ROM becomes more complex, as you're reading from it while... executing from it.


One downside I foresee is not being able to SIO the standalone .exe over the version in the ROM.
This just means shifting the ORG by 100kb or so.
Not even remotely a problem as I was doing that anyway.

Sure you lose a little flashable space, but honestly, not such a big deal.


All in all, I learned a ton though, so it's been good... but there's no point running the *whole* thing from ROM.