PBP Disk Image Format (Sony)

Members research, findings and information that can be useful towards the PlayStation 1.
Post Reply
User avatar
nocash
Verified
PSX Aficionado
PSX Aficionado
Posts: 541
Joined: Nov 12, 2012
Contact:

PBP Disk Image Format (Sony)

Post by nocash » August 27th, 2022, 11:42 pm

Discussing the PBP format was originally started in another thread (see this post viewtopic.php?p=21437#p21437 and scroll up to older posts).

Code: Select all

CDROM Disk Images PBP (Sony)
----------------------------

PBP Format (rev-engineered from homebrew DBALL.PBP)
  000000h 4    ID (00h,"PBP")
  000004h 4    Version? (10000h) (but, reportedly "always 100h or 1000100h")
  000008h 4    Offset of the file PARAM.SFO (28h)
  00000Ch 4    Offset of the file ICON0.PNG (3D8h)
  000010h 4    Offset of the file ICON1.PMF (3D8h) or ICON1.PNG
  000014h 4    Offset of the file PIC0.PNG  (3D8h) or UNKNOWN.PNG
  000018h 4    Offset of the file PIC1.PNG  (3D8h) or PICT1.PNG
  00001Ch 4    Offset of the file SND0.AT3  (3D8h)
  000020h 4    Offset of the file DATA.PSP  (3D8h)
  000024h 4    Offset of the file DATA.PSAR (10000h)
  000028h ..   PARAM.SFO file (zerofilled in homebrew PBP)
  0003D8h ..   PNG files etc  (zerofilled in homebrew PBP)
  010000h 0Ch  ID "PSISOIMG0000"
  01000Ch 4    PBP Size-10000h              (144740h)
  010010h 4    PBP Size-6420h (???)         (14E320h)
  010014h ..   Zerofilled
  010400h 0Bh  Game ID ("_SCUS_94476" for Hot Shots Golf 2)
  01040Bh ..   Zerofilled
  010800h A00h TOC List (0Ah-byte per entry, unused entries are zerofilled)
  011200h 20h  Zerofilled
  011220h 4    PBP Size-D2CFh (???)         (147471h)
  011224h 4    Zero
  011228h 4    Unknown (7FFh)
  01122Ch 11h  Game Name ("Hot Shots Golf",C2h,AEh,"2")
  01123Dh ..   Zerofilled
  014000h ..   Sector List (20h-byte per entry)
  ...     ..   Zerofilled
  110000h ..   Compressed sectors (starting with EDh,9Bh = non-ZLIB deflate?)
  15467Dh B8h  One extra compression block that is NOT in Sector List ???
  154735h 0Bh  Weird padding with ASCII "00000000000"
  154740h -    End of file
 TOC List (Subchannel Q with ADR=1 during Lead-In):
  000h 1   ADR/Control (eg. 41h=Data Track)
  001h 1   Track       (always 00h=Lead-in for all TOC List entries)
  002h 1   Point       (A0h, A1h, A2h, or Track 01h and up)      (BCD?)
  003h 3   Dummy MSF   (usually 00:00:00 or weirdly 00:02:01)    (BCD?)
  006h 1   Reserved    (00h)
  007h 3   Actual MSF  (or TOC info Point=A0h,A1h)               (BCD?)
 Example TOC (DBALL.PBP):
  41 00 A0 00 00 00 00 01 20 00   ;First Track (1) and Type=20h=CDROM-XA)
  41 00 A1 00 00 00 00 01 00 00   ;Last Track Number (1)
  41 00 A2 00 00 00 00 27 19 22   ;Lead-Out, uh at 27:19:22 in DBALL.PBP ???
  41 00 01 00 02 01 00 00 02 00   ;Track 1 at 00:02:00
  (remaining entries are zerofilled)
 Sector List:
  000h 4   Offset-110000h to Sector(N*10h)
  004h 2   Compressed size of Sector(N*10h+(0..0Fh))   ;9300h=uncompressed?
  006h 2   Zero (but, reportedly "usually 1... and 0 for the last entry")
  008h 10h Zero (but, reportedly "first 10h bytes of SHA1 sum of 10h sectors")
  018h 8   Zero (padding)
Data Compression format is unknown:
  Reportedly standard zlib ???
  but it doesn't have zlib or gzip header?
  maybe it's raw deflate`without header?
  one tool is using some kind of "sharp" compression?
  one tool is using some kind of "lz" compression?
Audio Compression format is unknown:
  ?
Multi-disc format is unknown:
  ?
Retail files have "PGD" encryption:
  ?

User avatar
nocash
Verified
PSX Aficionado
PSX Aficionado
Posts: 541
Joined: Nov 12, 2012
Contact:

Post by nocash » August 28th, 2022, 7:43 am

I've decompressed the first block in DBALL.PBP. The compression is raw Deflate (without any zlib header and adler checksum or the like).

And the result is... very disappointing: They've just compressed the 930h-byte sectors as-is (without filtering out the sector header, ECC, and EDC). It's just the same loveless compression approach as in .CDZ disc images.

And worse, the PBP header seems to be hardcoded to being about one Megabyte tall. In case of the DBALL homebrew, the compressed PBP file is actually bigger than the uncompressed CUE/BIN files.

Maybe the audio compression is doing something more useful, but to get there, one would apparently first need to get through the encryption layer in retail PBP files : /

User avatar
nocash
Verified
PSX Aficionado
PSX Aficionado
Posts: 541
Joined: Nov 12, 2012
Contact:

Post by nocash » September 3rd, 2022, 12:37 pm

I can now run the homebrew PBP files in no$psx. It was relative simple for dball.pbp, but the format of the TOC List is all different in psalm69, which required a handful of "special case adjustments" to get it working, too.

Code: Select all

 Example TOC (DBALL.PBP):
  41 00 A0 00 00 00 00 01 20 00   ;First Track (1) and Type=20h=CDROM-XA
  41 00 A1 00 00 00 00 01 00 00   ;Last Track Number (1)
  41 00 A2 00 00 00 00 27 19 22   ;Lead-Out, uh at 27:19:22 in DBALL.PBP ???
  41 00 01 00 02 01 00 00 02 00   ;Track 1 at 00:02:00
  (remaining entries are zerofilled)
 Example TOC (PSALM69.PBP):
  01 00 01 00 02 00 00 00 00 00   ;Track 1 as audio <-- why that ???
  01 00 02 02 37 44 00 00 00 00   ;Track 2 as audio
  01 00 03 03 25 45 00 00 00 00   ;Track 3 as audio
  41 00 01 00 02 01 00 00 02 00   ;Track 1 as data <-- listed last?
  (weirdly, most MM:SS:FF values are stored in byte[3..5] instead [7..9])
  (there are no point=A0h,A1h,A2h entries)
  (remaining entries are zerofilled)
Btw. which tool did you use to create the PBP files? And did you use different tools for dball and psalm69?
Oh, you seem to have uploaded two psalm69 pbp files (26th and 27th august 2022), I've tested only the older one yet (as usually, I am having trouble to access many https pages in win98).

The PBP format is really crap (at least the homebrew ones). Apart from the different TOC List formats, there seems to be no support for Index values. And no end of last track entry, or well there are several imperfect ways to compute the end of last track:
- use the Point A2h Lead-entry (but, it's wrong in dball.pbp and missing in psalm69)
- count number of sector list entries with nonzero size (but each entry represents 16 sectors)
- as workaround: one could additionally decompress last block and try to count number of used/valid sectors in it
- for extra confusion, there is a "later-than-last block" after the "last block" (but it is missing in Sector List)

null
Active PSXDEV User
Active PSXDEV User
Posts: 61
Joined: Dec 18, 2021
PlayStation Model: SCPH-1002

Post by null » September 3rd, 2022, 6:48 pm

nocash wrote: September 3rd, 2022, 12:37 pmBtw. which tool did you use to create the PBP files? And did you use different tools for dball and psalm69?
It's FPse64 for Android made by (Italian devs?) Schtruck/LDchen.

http://play.google.com/store/apps/detai ... tor.fpse64

No, I used the same tool.

Right now there's no possible way to this on PC without including the specific license ID no. of the game (it needs to match with the game and you can't input your own customizable license ID) and you need to fill those PSP required files (PARAM.SFO/ICON0.PNG/ICON1.PMF or ICON1.PNG/PIC0.PNG or UNKNOWN.PNG/PIC1.PNG or PICT1.PNG/SND0.AT3). But I don't know how to do it again because it has been over a decade since I did that and uninstalled all those converters on my PC, sorry.

For iPhone users, there's no possible way do this as of now.

ePSXe doesn't have a built-in PBP compressor.
DuckStation doesn't have it also.
nocash wrote: September 3rd, 2022, 12:37 pmOh, you seem to have uploaded two psalm69 pbp files (26th and 27th august 2022), I've tested only the older one yet (as usually, I am having trouble to access many https pages in win98).
No, they're not the same. The August 26, 2022, uses PBP. It's the first PBP file with multi tracks. While the August 27, 2022 one uses CHD. It's the CHD I made using the latest CHDMAN v0.246.

Anyway, I edited one of the .CUE file there before I converted it to PBP and that game is Psalm69Demo. I renamed it to dball and the text inside its .CUE file because the PBP compressor can't recognized multi track demos with unknown license ID no.
Last edited by null on September 3rd, 2022, 7:12 pm, edited 3 times in total.

null
Active PSXDEV User
Active PSXDEV User
Posts: 61
Joined: Dec 18, 2021
PlayStation Model: SCPH-1002

Post by null » September 3rd, 2022, 6:54 pm

nocash wrote: September 3rd, 2022, 12:37 pmI am having trouble to access many https pages in win98.
Can your PC merge separated by parts deflate ZIP files? Or decompress LZMA 7Z files?

null
Active PSXDEV User
Active PSXDEV User
Posts: 61
Joined: Dec 18, 2021
PlayStation Model: SCPH-1002

Post by null » September 3rd, 2022, 7:40 pm

Any Windows/DOS tools you know that can convert .CUE/.BIN or .ISO files to .PBP files directly without needing those prerequisites?

null
Active PSXDEV User
Active PSXDEV User
Posts: 61
Joined: Dec 18, 2021
PlayStation Model: SCPH-1002

Post by null » September 3rd, 2022, 11:06 pm

There's one I found a public cmd tool with source code but it looks like it's too buggy to use because even if the app tells it completed the processes it'll always gives 0 byte file.

Try that one maybe it could work on your PC.

http://github.com/RupertAvery/PSXPackager

null
Active PSXDEV User
Active PSXDEV User
Posts: 61
Joined: Dec 18, 2021
PlayStation Model: SCPH-1002

Post by null » September 7th, 2022, 4:42 pm

Do you want a PBP header copy of one of the commercial retail PSN Classics Eboot? I have the Bowling North America NPUJ01288 with 14 audio tracks?

It sure works on POPS (short for PlayStation On PSP System), Sony's official PSone Classics emulator native to their PSP/PSVITA/PS2/PS3. I don't know why they called it POPS even for their other consoles?

For emulators, ePSXe doesn't support PSP Eboot for POPS. In DuckStation, it needs unencrypting first so unpack it then convert it into a PSX compatible PBP format. And the last one FPse64, it works without any modification and doesn't need any unencrypting/unpacking to play the EBOOT.PBP (no need for the KEYS.BIN/DOCUMENT.DAT)

Status:
POPS r13 (PS2) - working
POPS 6.60 (PSP) - working
POPS 4.82 (PS3) - working
POPS 2.60 (PSVita) - working
ePSXe - not working
DuckStation - working but needs conversion
FPse64 - working

heeltrick
What is PSXDEV?
What is PSXDEV?
Posts: 2
Joined: Sep 13, 2023
PlayStation Model: scph-70000
Discord: heeltrick
Location: Japan

Post by heeltrick » September 13th, 2023, 1:35 pm

Not sure if you guys are still looking into it, but fwiw I spent some time trying to reverse engineer the ATRAC3 audio encoding and released a version of psxtract that is now able to fully recover all tracks including audio from retail EBOOT.PBP files. I'm leaning on the at3tool to do the actual decoding as it does the best job in terms of getting very close to original quality, but ffmpeg works as well once the proper headers are generated for each audio track. The code is here if you're interested:

https://github.com/has207/psxtract-2/bl ... xtract.cpp

User avatar
nocash
Verified
PSX Aficionado
PSX Aficionado
Posts: 541
Joined: Nov 12, 2012
Contact:

Post by nocash » September 17th, 2023, 5:22 pm

Very good to know that the audio format is called ATRAC3, thanks!

I've never heard about that format, I've looked up the wikipedia article, but I am still not sure what it is (concerning decompression speed, quality, and compression ratio, compared to other lossy formats like ADPCM or MP3).

Oh, and do you know how consoles like PSP are decompressing that format, by software or hardware?

heeltrick
What is PSXDEV?
What is PSXDEV?
Posts: 2
Joined: Sep 13, 2023
PlayStation Model: scph-70000
Discord: heeltrick
Location: Japan

Post by heeltrick » October 6th, 2023, 11:02 am

I'm not an expert on this but afaik sony does the decoding in hardware on the consoles. The format was popular for their minidiscs as well, and it's apparently very good quality-wise, at least if you use their tools to decompress. Open source implementation in ffmpeg is notably worse based on my limited testing.. There's a bit of a technical/marketing overview from sony:

https://www.minidisc.org/www.sony.net/P ... index.html

null
Active PSXDEV User
Active PSXDEV User
Posts: 61
Joined: Dec 18, 2021
PlayStation Model: SCPH-1002

Post by null » October 26th, 2023, 3:21 am

nocash wrote: How consoles like PSP are decompressing that format, by software or hardware?
As of now, no hackers fully reverse engineered the PSP's Tachyon, only partially.

I guess the decoding is in Virtual Mobile Engine DSP, inside Tachyon (the PSP main CPU SoC IC). Not only PSP uses this even the SONY Walkman NW-MS70D has a built-in one. Lots of SONY Walkmans supports ATRAC3/ATRAC3+ decoding.
nocash wrote: Decompression speed, quality, and compression ratio, compared to other lossy formats like ADPCM or MP3?
There's some old test made by SONY with charts but I guess it's a little bias.

Code: Select all

                           CD (LPCM)  ATRAC1   ATRAC3   ATRAC3plus  MP3
BITRATE QUALITY             1411kbps   N/A      N/A      256kbps     N/A
STD BITRATE LOSSY           N/A        292kbps  132kbps  64kbps      128kbps
COMPRESSION RATIO           1          1/5      1/10     1/20        1/11
YEAR                        N/A        1992     2000     2004        1991
References:
PSPTEK - http://daifukkat.su/docs/psptek/
Tachyon - http://www.psdevwiki.com/psp/Tachyon
at3plusdecoder.dll - http://github.com/hrydgard/ppsspp/pull/997
atrac3plus.c - http://raw.githubusercontent.com/FFmpeg ... rac3plus.c
VMEDSP - http://www.copetti.org/writings/console ... -portable/
ATRAC3PLUS FAQS - http://www.minidisc.org/minidisc_faq.html
PSP HARDWARE DOCS - http://uofw.github.io/upspd/docs/
SAMPLE TEST - http://hydrogenaud.io/index.php/topic,24735.0.html
http://www.edepot.com/reviews_sony_psp.html

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests