New open-source apps to upload PSX-EXE and external data

Start a work log and update it occasionally with your projects progress
Post Reply
Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

New open-source apps to upload PSX-EXE and external data

Post by Xavi92 » July 21st, 2017, 9:23 am

Hello everyone,

I bought one of Dan's PSX serial-to-USB cables a few months ago and wanted to get them working with my PSXSDK demos. However, it looks like PSXSERIAL jumps to a memory address which does not correspond with PSX-EXE's initial program counter. So I wrote my own version of it with a GUI written with Qt and PSXSDK, and release all the source code under GPL on Github:

PC-side application (Windows 32-bit build also available):
https://github.com/XaviDCR92/QPSXSerial

PSX-side application:
https://github.com/XaviDCR92/OpenSend

Procedure:
1. Open QPSXSerial and select desired folder where your PSX-EXE and SYSTEM.CNF files are located, with the same folder/file structure that you would burn into a CD.
2. Turn on your PSX with an image of OpenSend.
3. Click "Send to PSX!" button. PC will automatically detect connection with the PSX and will dump necessary data into the desired memory address (indicated inside the PSX-EXE header).

The interesting thing about this application, contrarily to PSXSERIAL and hitmen's SEND, is the capability of sending external files e.g.: TIM/VAG... via serial port. That would remove the necessity of statically compiling all external data into the executable. This requires your demo to interact with QPSXSerial accordingly. There is a semi-working example on my video game "Airport":
https://github.com/XaviDCR92/Airport

It's in a very preliminar state and transferring PSX-EXE data is painfully slow, but gets the job done. There is still a lot of room for improvement, that's why I have decided to release the source code.

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » July 25th, 2017, 7:41 am

I've made some improvements on the tools. Performance has been improved when transferring PSX-EXE data. Give it a try! :)

Image
Image
Image

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » July 27th, 2017, 7:49 am

Live demo :)
[youtube]http://www.youtube.com/watch?v=YyorqauWiPw[/youtube]
Last edited by Xavi92 on January 5th, 2018, 4:06 am, edited 1 time in total.

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » August 6th, 2017, 2:50 am

By popular petition, QPSXSerial is now CLI-compatible! Check it out at the link below:
https://github.com/XaviDCR92/QPSXSerial/

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » August 12th, 2017, 7:15 am

I hadn't realize BIN/CUE images for OpenSend were not uploaded to the repository! Find them on the following link:

https://github.com/XaviDCR92/OpenSend/tree/master/Bin

Enjoy! :)

User avatar
Shendo
Verified
C Programming Expert
C Programming Expert
Posts: 250
Joined: Mar 21, 2012
I am a: Programmer
Motto: Never settle
PlayStation Model: SCPH-7502
Discord: ShendoXT
Location: Croatia, EU

Post by Shendo » September 12th, 2017, 3:32 am

Awesome.

I was thinking, can you add some kind of checksum while uploading data chunks to memory?
That way if you are testing your code and data is already in memory from last time
upload function can just skip the file and go to the next one.
Or if a file is slightly modified from the last time only affected chunks will be uploaded.

I assume you'd have to be careful about memory management while coding but since afaik ps1
doesn't have a clue about virtual memory it should be doable.
Dev console: SCPH-7502, FreePSXBoot, CH340 serial cable.

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » September 13th, 2017, 5:13 pm

That sounds pretty interesting. The problem with modifying OpenSend (PSX client for QPSXSerial) is that it cannot be tested under emulator so I end up wasting tons of CDs until I get it working. But I'll take your suggestions into account. :)

User avatar
nocash
Verified
PSX Aficionado
PSX Aficionado
Posts: 541
Joined: Nov 12, 2012
Contact:

Post by nocash » October 1st, 2017, 5:03 am

Xavi92 wrote:it cannot be tested under emulator so I end up wasting tons of CDs until I get it working.
Huh? Isn't the cable upload intended to test things without CDs?

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » October 3rd, 2017, 9:39 am

When the application was first developed, there was a chicken-and-egg problem, that's why I had to spent some CDs until it was working OK. :) However, there is an additional problem: OpenSend is first loaded on a specific memory address (located near the right end of PSX's 2MB RAM range) so that guest applications can use most of PSX RAM, starting from 0x80010000 (right after the 64 KB occuped by the BIOS).
If OpenSend needs to be developed further, then it cannot use the same memory address than the host application! Surely though, it could be temporarily moved to a lower position so it doesn't collide against the host application. :shrug

User avatar
Shadow
Verified
Admin / PSXDEV
Admin / PSXDEV
Posts: 2670
Joined: Dec 31, 2012
PlayStation Model: H2000/5502
Discord: Shadow^PSXDEV

Post by Shadow » October 3rd, 2017, 3:45 pm

Technically there is room in the kernel that can be used to execute code, which will give you the full 2 MB of address space. This is what CAETLA did. Problem is, you'll need to develop 'OpenSend' in raw assembler and make it as small as possible.

I also got your PM, and yes, I still have the source to PSXSERIAL. I really don't want to share it at the moment though since I will be making it in assembler sometime to have the ability to use the full 2 MB addressing, and it's not always good to have the same product being released under a different name (just confuses people as per what one they should download and use). I think your version makes it now the fourth type of program you can use :P
Development Console: SCPH-5502 with 8MB RAM, MM3 Modchip, PAL 60 Colour Modification (for NTSC), PSIO Switch Board, DB-9 breakout headers for both RGB and Serial output and an Xplorer with CAETLA 0.34.

PlayStation Development PC: Windows 98 SE, Pentium 3 at 400MHz, 128MB SDRAM, DTL-H2000, DTL-H2010, DTL-H201A, DTL-S2020 (with 4GB SCSI-2 HDD), 21" Sony G420, CD-R burner, 3.25" and 5.25" Floppy Diskette Drives, ZIP 100 Diskette Drive and an IBM Model M keyboard.

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » October 4th, 2017, 6:35 am

Shadow wrote:I think your version makes it now the fourth type of program you can use :P
Not really - OpenSend and QPSXSerial not only allow uploading PSX-EXEs as other applications e.g.: PSXSERIAL, hitserial or JavaPSXSend already do, but also provide an interface to upload any data the user wants to the consoloe, as long as the game is using such interface (take my video game "Airport" as an example). This breaks the limitation of bundling all data into the PSX-EXE.On the other hand, there is also a small console which outputs all printf() calls made from the console.

So no, this is not yet another application to upload PSX-EXEs. :P

rama3
Verified
/// PSXDEV | ELITE ///
/// PSXDEV | ELITE ///
Posts: 510
Joined: Apr 16, 2017

Post by rama3 » October 6th, 2017, 2:06 am

Shadow is probably concerned about 'protocol fragmentation' and resulting confusion for newcommers on what kind of software to use.
In that regard, I think it would be best to develop this version into the recommended standard.
The additional features are well worth it. I can't imagine why uploading additional data was never much considered ;p

If memory limitations won't allow it, the additional features could be advertised in a feature byte.
All other client / psx side software could be made to read such a feature byte and then respond properly.
A pc side SIO printf console is really nice, for example.
(Sidenote: Does that console work with SIO prints from nocash's new kernel clone?)

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » October 6th, 2017, 10:21 am

rama3 wrote:Shadow is probably concerned about 'protocol fragmentation' and resulting confusion for newcommers on what kind of software to use.
In that regard, I think it would be best to develop this version into the recommended standard.
PSXSERIAL is a closed source application, hitserial relies on old DOS headers, and other applications do not support uploading external data. The reason why I decided to release my tools under the GPL license was to include everyone's contributions so as to create the ultimate tool for PC<->PSX communication, from what others can read and learn. That's essentially the whole point about open-source software.
rama3 wrote:(Sidenote: Does that console work with SIO prints from nocash's new kernel clone?)
It was only tested with a SCPH-5502 with stock BIOS. However, you can always test it on your own and dig into its source code if you want to contribute. :)

rama3
Verified
/// PSXDEV | ELITE ///
/// PSXDEV | ELITE ///
Posts: 510
Joined: Apr 16, 2017

Post by rama3 » October 6th, 2017, 11:41 am

Sure, once I actually build a SIO cable. Parts are all here already, just need to find the time :)

(I didn't know that the other tools were closed source or very old. So that's a huge plus on your modern alternative!)

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » October 6th, 2017, 4:25 pm

rama3 wrote:(I didn't know that the other tools were closed source or very old. So that's a huge plus on your modern alternative!)
As far as I'm concerned, MCLaunch (which is also a great tool) is the only open-source alternative apart from QPSXSerial/OpenSend. Besides, QPSXSerial is the only tool to include both GUI/CLI interfaces. :)
Shadow wrote:Technically there is room in the kernel that can be used to execute code, which will give you the full 2 MB of address space. This is what CAETLA did. Problem is, you'll need to develop 'OpenSend' in raw assembler and make it as small as possible.
I don't think that's the target for OpenSend at all. Even if well-structured assembly can be still understandable for most programmers, OpenSend was written in plain C so everyone can contribute and understand how the tool works. Let's be realistic: unless you are creating the next big thing (e.g.: Orion's RPG-J game) or simply doing it wrong, it's much unlikely your game is to take more than ~1,50 MB RAM. In fact, not even my video game(Airport), which development status isn't far from finished, is currently exceeding the 500 KB limit.

Orion_
Verified
Legendary Programmer
Legendary Programmer
Posts: 240
Joined: Aug 13, 2012
I am a: Programmer
PlayStation Model: Net Yaroze
Location: France
Contact:

Post by Orion_ » January 29th, 2018, 7:33 pm

I tried this tool because I needed to upload a program to my PSX without using the AR port (no Xplorer FX)
I made a CD with Loser PSX Menu selector, and when launching PSX Serial I get a black screen, maybe they are using the same memory area (end of ram), so I tried to launch PSX Serial by sending it to caetla on my Xplorer FX, and same thing, black screen, I fear to waste a CD-R just for this ...
so before doing that, I just want to know if this tool is working with the Serial cable of the Net Yaroze ?

On the PC side, I really don't understand why the app is using 40 Mbytes of DLL just for this little tiny program ....
Win32 API is really simple to use, why using QT ?
or at least you could use just a CLI interface and would take only a few Kbytes...
Also, your app is missing this DLL "libgcc_s_dw2-1.dll" to run properly.

Another option you should add to your utility is to be able to send an EXE, not only choose from a folder..
Retro game development on Playstation and other consoles http://orionsoft.free.fr/

Xavi92
Verified
C Programming Expert
C Programming Expert
Posts: 161
Joined: Oct 06, 2012
PlayStation Model: SCPH-5502
Contact:

Post by Xavi92 » January 30th, 2018, 11:27 pm

Orion_ wrote: January 29th, 2018, 7:33 pm I tried this tool because I needed to upload a program to my PSX without using the AR port (no Xplorer FX)
I made a CD with Loser PSX Menu selector, and when launching PSX Serial I get a black screen, maybe they are using the same memory area (end of ram), so I tried to launch PSX Serial by sending it to caetla on my Xplorer FX, and same thing, black screen, I fear to waste a CD-R just for this ...
so before doing that, I just want to know if this tool is working with the Serial cable of the Net Yaroze ?
OpenSend starts at memory address 0x801A0000, but I don't know if this could collide with other loaders or fail when used together with XPlorer FX cartridge. I tested it using a CD-ROM on a SCPH-5502 with one the cables made by danhans, which IIRC just feature a USB-to-UART adapter (probably a FTDI). I cannot ensure this is working with Net Yaroze serial cables as I don't own one.
_Orion wrote:On the PC side, I really don't understand why the app is using 40 Mbytes of DLL just for this little tiny program ....
Win32 API is really simple to use, why using QT ?
or at least you could use just a CLI interface and would take only a few Kbytes...
Qt ensures easy portability between platforms (Linux, Windows and Mac) without any modifications. Also, QPSXSerial actually supports both GUI/CLI modes - just call "QPSXSerial.ese --help" and you will see the possible options. I know Qt DLL files are huge, but I can't do anything about it. Probably Python or Java would be better suited for this task.
_Orion wrote:Also, your app is missing this DLL "libgcc_s_dw2-1.dll" to run properly.
You are totally right. I'll fix that ASAP.
_Orion wrote:Another option you should add to your utility is to be able to send an EXE, not only choose from a folder..
I designed QPSXSerial to depend on SYSTEM.CNF file, as it points to the desired EXE file, which is useful if more than EXE file is present. That could be changed easily, though.

ycelik
Curious PSXDEV User
Curious PSXDEV User
Posts: 20
Joined: Sep 02, 2018

Post by ycelik » December 9th, 2018, 10:20 pm

_Orion wrote:On the PC side, I really don't understand why the app is using 40 Mbytes of DLL just for this little tiny program ....
Win32 API is really simple to use, why using QT ?
or at least you could use just a CLI interface and would take only a few Kbytes...
Xavi92 wrote:Qt ensures easy portability between platforms (Linux, Windows and Mac) without any modifications. Also, QPSXSerial actually supports both GUI/CLI modes - just call "QPSXSerial.ese --help" and you will see the possible options. I know Qt DLL files are huge, but I can't do anything about it. Probably Python or Java would be better suited for this task.
Yeah, I can vouch for this, since I have just compiled everything for the Mac.
On the other hand, despite working really good on my WIN10 machine, I could not get PSXSERIAL.exe to see my comport in Wine on my Mac. Moreover, since it is also a CLI app, I would be able to configure a Visual Code build task that could automatically upload my files to the PSX on compilation.

Post Reply

Who is online

Users browsing this forum: No registered users and 5 guests