Thesis about a PS1 Game Engine

[maltebp]

Hello peeps!

My name is Malte, and I'm pretty new around here, so I thought I'd say "hi" and talk a bit about what I'm up to

The short version
I'm doing a basic game engine/framework for the PlayStation 1 for my final Master's thesis.

The (very) long version
I'm taking a Master's degree in Games Tech, and atm I'm doing my thesis. I want to be an engine programmer, so I thought I'd do something related for my thesis. Honestly, I just wanted to program an engine, but a thesis requires some kind "research". So I ended up deciding to try and create and test the viability of general-purpose 3D game engine for the PlayStation 1.

I'm planning on testing the viability by measuring the performance of the engine, but in pretty bad way... You see, because "hardware hacking" (e.g. modchips) is a bit of a legally grey area where I'm from, my supervisor won't approve the use of it for the thesis (formally, at least). So the "research question" is instead going to be answered by testing the performance in an emulator. Yes, I know this is super inaccurate, but as long supervisor approves it I can get to tinker with my engine. Also, it still gives some notion of the capabilities (can the engine handle 1 million game objects or 1000).

I have no prior experience with PlayStation development, and only a bit of experience with really low-level programming (assembly) and hardware, so this has obviously taken some getting used to. Already spent my fair share (at least more than I anticipated) trying to understand the hardware and low-level stuff. I do however, have some experience in game and game engine development, so I hope it will be "easier" once I get the basics down and start moving into my own "turf".

Status
Throughout February, I've been creating a small game in order to understand the technicalities of the PlayStation and the psn00bsdk (which I'm planning to use as a base for my engine). Some kind people on the Discord server has helped me out with various questions, so thanks a lot for that.

I just finished that small project yesterday, and you can see a demo here:


Disclaimer: it's shit gameplay (is it even a game?) and I play with one hand.

It's pretty rudimentary, but I got to test directional lighting, 3D mesh rendering, textures, sounds, music, sprite rendering and debug text.

I'm definitely not an electrical engineer, and I personally failed at installing a modchip into my PlayStation using a cheap soldering iron. A real nice guy helped me out though, so it's all set up (yay!). I did figure out how to create a serial-to-usb cable, so with Unirom and nops I have a nice work-flow with the actual hardware.

Atm, I'm using PCSX Redux as my go-to emulator, as that seem to be the best in terms of debugging and development capabilities (the GDB server is really, really nice).

So now, I'm basically moving on to create the actual engine.

Goal
I'm really not planning on this engine being any kind of marvel as I'm all alone and have fairly limited time. And I expect I need to spend significant time wrestling with the PlayStation and psn00bsdk. I'm planning on the following features:
- Game object system (approach undecided)
- Renderering of textured and lit 3D meshes
- Asset system (maybe including packing)
- Various base utility (fixed-point, strings, list, vector, matrices, map etc.)
- Rudimentary collision system
- Sound system
- Text system
- Basic 2D image system

This is not much, but it might be a stepping stone for something more interesting after my thesis (we'll see if I'll continue working on it).

The plan is to get it in a state where I can use for a game jam in mid April, which of course will go so smooth that we'll have the best game ever, and people will ditch all other engines in favor of this brilliant piece of software.

I've only found one other attempt/talk of a PSX engine (the Lameguy64's engine), so I don't know if this project might be of interest to someone. Maybe just as a foundation or inspiration for something more serious? Not expecting it, but thought I'd might share it anyway. Unfortunately, I can't share the source-code before the thesis has ended (as that would make me plagiarize myself), but I'll definitely share it once the project is done!

Any tips, tricks, opinions, recommendations or questions are very much welcome!

[nocash]

The boot screen where it is saying "d" and "d" without PS logo, is that because you forgot to include the licence/logo sectors, or is it a hardware glitch (that also happens on retail discs)?

And the game engine, are you writing own GTE code, creating your own linked lists, and doing the GPU DMAs on your own - all without using SDK functions for that?

[maltebp]

I don't know about the boot screen actually - haven't been (bothered) looking into ut yet. Pretty sure it doesn't happen on retail discs, but have to double check that. I'm using psn00bsdk which is using MKPSXISO which I'd assume would work properly, so maybe it's on my end?

For this project I'm not planning on writing any GTE or GPU code all on my own that doesn't already exist in the SDK I'm using (psn00bsdk). For the long term though, I would definitely consider moving away from some of the SDK APIs where I might be able to gain some performance improvements because I can implement them for my specific domain.
I am implementing my own very basic containers though. But that is mostly because I don't want to have to try and incorporate an external library (e.g. EASTL) and having to set it up to run on the PlayStation. Also, I think I can implement the most important bits in a performant enough way pretty easily (the bulk of STL seem to me to be in the flexibility and all the little details catching a lot of edge cases).

[gwald]

I dont see what's new or challenges anything about this thesis?
There have been many games that used the same "engine" on PSX, I know the die hard trilogy engine is pretty generic (with fake 3D people) and it was used on another different game.

There are PS games that have source code available too, and decompilation is a thing now.

If the aim of the thesis is to make a generic game engine, I dont doubt it can be done (like DHT), but because of the limitation it will most likely be big and slow, and like DHT, look very "samey".
The PSX is too limited to do a generic engine, instead a proven game's engine was typically reused for other games.

Your feature list reminds me of libGS, Sony's generic 3D & 2D high level lib, which was used mostly in early games and in budget games, because it wasn't performant, ie slow.
libGS doesn't do 3D collision detection (or physics) because not every game requires it and often very custom, so it's a waste of space, more or less like the other unused features.

I doubt you can single handed do in a few weeks what a team of paid pro's did over a few months... and now reversed engineered and extended for everyone to use.
https://github.com/OpenDriver2


I dont mean to talk you out of it, just making you aware of a few things.
I think the only reason why anyone would use some PSX related engine or framework is if it offered portability, ie making something on it will run on PSX and also many other platforms... like PC/Mac/HTML/Android/ios/SDL1/SDL2/3DO/Saturn/DC/GC/etc/etc/etc/etc

I did just that with 2D, it wasn't hard. https://github.com/gwald/yarmico2/tree/ ... ary-builds
Doing the same with 3D is harder, but still doable.

[maltebp]

Thanks for your feedback - it sheds some light on things for me.

You speak of the DHT engine being pretty generic, but I can't find any sources describing that in more detail. Would love to read about it if you have something. I don't doubt that it was used for the trilogy, and perhaps a few offshoots of similar games, but that does not really constitute "general purpose" for me. And it seem to me that that is the general historic trend ("engines" were made for one game/series and a couple of similar games).

I actually had no idea that that was what libGS was. I have seen the name, but didn't other looking into what it actually was. Sounds like something I should maybe look into for some perspective/inspiration. Though, I don't see how unused features of an API would be a waste of space? To my understanding, a sensible compiler and engine setup should be able to get rid of that.

I don't agree that the only reason people would want to use a PSX engine would be for portability across many consoles. Speaking with peers and colleagues who involves both designers and programmers, people seem to be find an interest in this whole project and in the PSX development. I think a more accessible way of developing games for the PlayStation would still be of interest to some. In particular, I think there might be an interest among game jammers and solo home-brew developers, who don't intend to make epic-scale games anyway.

I don't doubt that a general-purpose game engine will definitely have to do some performance trade-offs to provide more generalized systems and workflows. But what I'm wondering is: how much exactly? Is it feasible to make anything of interest with a general-purpose framework? and in that case, how capable would such a framework be? Asking these questions is basically the academic approach of my thesis. We have ideas of what the result my look like, but because those are just notions (despite yours being more qualified than mine) testing it out and discussing the hows and whys the results end up like they do still has "academic" value. At least enough value for me to get this approved as a thesis, so that I can work on developing an engine (which is what I really want to do)

A clarification that might be worth making is that I develop this with the idea that a more accessible workflow should be laid on top (i.e. an actual editor), but that I won't actually have time for that in the thesis. So the idea is not for it simply to be a coding library (like what libGS seem to be). I think this provides ways of auto-generating and hard-coding a lot of things to provide some performance benefits while still being able to expose them in a user-friendly manner. That might be me being overoptimistic though.

Sorry for the long response Your comment helped reflect on some stuff and that was just unloaded here.

[gwald]

No problem, well, DHT was a FPS (on rails), a driving game and 3rd person top down... and doing it all pretty well, to me that's pretty general purpose and flexible... it's not perfect, characters look quirky (made from billboards) but it could have been used to make pretty much any 3D game, ie GTA in 3D, same with the driver2 engine... but DHT came early and Driver2 came later thus looks better and has real 3D people, but I would guess DHT could have more animated characters on screen being a lot lighter in polys, DHT also had a built in level editor for DH2. Tomb Raiders engine... Mario64 maybe etc etc and also on github.

They were coded for the purpose of 3D animated characters, objects and world, that's pretty general IMO and also proven using specific limited features... and IMO better then a generic (what you call general purpose) game engine... which has to be as generic as possible and cover every use case possible... I think while it's possible, like libGS, the aim is completely different to a specific game engine like above.

What's the difference?
A game engine's aim is to have small file format for assets as possible, libGS is the opposite, everything the playstation can do, the file formats has to cover it, thus bloated and slower to process, have you used Collada?

Rendering, games have specific and limited rendering, a generic engine/framework has to render every poly with every possible combination... texture, lights, fog, RGB, etc etc

The compiler isn't going to compile out bloated functions that processes a big file format or complicated rendering lol.
And on a system with only 2MB that's a very big deal.

I"m sure there are many other differences, but that's the main ones of the top of my head.

Also, libGS was use to ease developers into 3D, it had 3D app plugins and converters for artist and it saved programmers from starting from scratch and coding a game engine. It was also small (in functions not size) so less tech doc's to read, games could come out quicker, and they did... libgs was important at the start and for budget games.

LibGS was good enough that Sony made Net Yaroze limited to libGS and other high level functions, audio/cdrom/memcard/pads/etc, and turned it into a hobbyist gamedev program.

Will Bott was quoted talking about Net Yaroze in a Next Gen 25 article: Videogame school and build a better game: "If you can't make a game with the tools that Sony provides, you're an idiot and don't belong in the business"
I love that quote, that's how we spoke back in the 90's... no BS, real and very motivating!!

Net Yaroze uses TMD (the do everything bloated format) exclusively for 3D, but in the pro version there are more optimized formats HMD, PMD and the same with 2D, so it didn't compete with the dev's making budget games, I guess.

" Is it feasible to make anything of interest with a general-purpose framework? and in that case, how capable would such a framework be?"

That is very vague wording but here is a list of a few games made with libGS:
https://problemkaputt.de/psxspx-cdrom-f ... d-sony.htm
Also Japanese budget games typically used libGS.

I'm all for game engine development, I think a simple (with limited set of features) engine is doable, educational and making a game on your own tech is always easier.

The internet being the hive mind of the dumbest with the most time, always tries to talk people out of engine dev with the same BS arguments... you cant make unity3D or UE so dont bother.... which is completely not the point of doing it in the first place.

But in your case that's what you want to do with a 'general-purpose' (what I call generic), but on 30 year old hardware lol
Have your peers build a generic game engine? what does the file format look like? is it big? if it's on github post a link.

You could also ask around on the n64dev scene, its 2x the HW of playstation but see if a generic engine is doable on 4/8MB rather then the PS's 2MB. And TMK N64 doesn't have a libGS equivalent.

Or target both PSX/N64, engine programmers like reading tech docs and hardware spec's
And add Saturn for the holly trifecta
oh and SDL because it's easier working on PC

[maltebp]

It sounds like they had some cool things in DHT, and that those parts would definitely be part of a general-purpose engine. But to me is also a question about how much of this is exposed to a developer to change - i.e. is it all hardcoded in a way that the user gets very little say in how they can model that 3D world and game-play, or does it allow for many different setups? I would love to see the tools and workflows they had for those games.

It's true that if you make a single monolithic system that wants to do everything the compiler probably isn't going to help you a whole lot, because it cannot know what feature are used or not if it's data-driven. I'm rather trying to think in ways of compositing the engine of different blocks that can be toggled on/off, preferably at a pre-processing step to leave them out of the code-base entirely. I may be dreaming and totally over-optimistic here, but I think you could expose some interesting features this way. I'm not sure how big of a deal it might be, but there is also the big advantage that the workstations today are significantly faster relative to target machine than they used to be, giving us a bigger budget in what we can actually pre-process.

You're pointing something out though, that I might not be paying enough attention to: the size of the code. Not really used to having that being a limiting factor, so it's a good point to keep an eye out for that.

Regardless, you are of course right about the more specific engine would be easier and faster. Both in it's theoretical potential, but also in the practice case - given the same time you can probably write something more performant for a specific case compared to the general case. I'm not disputing this - I'm just wondering if there is a potential for something more general-purpose and easier accessible. Easier accessible than even what I think LibGS is. Because, forgive me if I'm mistaken, but LibGS is "just" an SDK/framework and entirely code based, right?

Though, it definitely sound like I can't get around having a look at LibGS didn't realize it was the SDK exposed to Net Yaroze either.

Not sure what to think about that quote from Will Bott. I think it falls beside the point of my project here.

I'm not interested in moving beyond multiple platforms - tackling a single platform unknown to me, with hardware limits like this is more than enough for now That would also be moving even more into an even more general-purpose engine, no? It's an interesting problem to consider, for sure though. Maybe some other time that might peak my interest.

I completely agree with the a lot of people completely missing the point about hobbyist game engine dev. Though, I disagree that I'm trying to make Unity3D or Unreal. Sure that's the *kind* of engine I'm thinking about, but I think they have moved far beyond what is "necessary" for such an engine, and provide a ton of extra tools and features. I think you can still talk of a general-purpose engine while it being much smaller in scope.

For me, I personally though just want to develop an engine, because I learn a lot from it and I think it's really fun. But the fact that I personally think the project have the potential to be useful to some one else (even though that's probably just me day-dreaming), just makes it that more motivating.

I did a quick and dirty model format and a compiler to go with it that compiles from *OBJ* because I needed it for a short jam last Friday. It was rather buggy and not thought through a whole lot- it basically just took the vertices, normals and indices and compiled them to a binary format. I'd love to share my project, but as I mentioned, I won't be able to until I have handed in my project.

[gwald]

They filmed the making of DHT: https://www.youtube.com/watch?v=HVgokAf ... 4lCd11n20R

I think the in game level editor would have been used internally, it's just a guess tho, but to program that feature to me suggest that it was, eg the game testers were also fixing levels.
From my poor memory, I recall thinking it was pretty sophisticated, and it saved to the memcard.
On the disc it's limited to DH2 (FPS on rails) but maybe it had the RAM space for the editor while the other games did not, just a guess, the test units had 8MB's so no RAM limits.

Like this, he said he's retexturing infront of a blue test unit, but not sure if he's actually using it tho: https://youtu.be/_E6blKdyB9k?list=PL2-- ... n20R&t=320

I cant speak to that idea about the preprocessor, or even using C++ features, no idea how that would work.

Yes, libGS is a pure software API that sits on the rest of the SDK, a black box as their third party dev's would call it
I've asked the driver2 reverse engineer guy if he will do libGS, nope!

While the pro LibGS was smaller then understanding the detail of the hardware and low level API's, it isn't what I would call a small API, it does a bit.... 2D/3D assets, scene and animation, and as you said the compiler only adds what's used.

The Net Yaroze in comparison is a very cut down (and simplified at places) version of libGS (eg 2D cell background is included but not the animation), and the fact that Net Yaroze requires the libPS in RAM, which is libGS+other libs in a single DLL like file. 500KB
I don't think this was done to limit RAM use, which it really does, 1/4 of it... but more to make the executables smaller to send over the serial cable, which back then wasn't necessary very reliable at times.

Here is a copy of libps.h, most of the Net Yaroze API, https://github.com/gwald/psyq_to_netyar ... bps.h#L394

In hindsight, I think it also serves as a means of focusing on what's there in RAM than rather rolling your own or low level code etc, which back in the 90's was the common thinking, not like now, where everyone wants an engine to do everything for them


Thus Bill's quote.... it's the core of our discussion.
Net Yaroze has a very small API in functions but not in functionality necessarily, as most would commonly assume.
You would have to research to understand what I mean.

IMO Net Yaroze was equivalent to unity3D of the 90's in many ways... but very few people took it seriously. It was a great tool for what it was and still is Bill was right to call devs idiots if they couldn't make something nice with a simple tool. Sounds like a stupid thing to say in 2024 but in 1997 we all had high standards of ourselves, ie visuals and gameplay, nobody wanted to make something small or simple... or crap... now most want to make crap with the least of effort/time, and sell it - Tangent, but have you thought about that? if most people are happy with small, lesser effort games, what will happen to your industry? eg big studios? consoles? I think it will go like Hollywood, and the industry will contract, smaller and smaller.


Even if Sony made the NY simpler, or created better plugins or more docs, tuts or helper code etc etc the results would have been the same, everyone just wanted low level access to HW and they were going to do everything themselves
Idiot was too polite

But you think that mentality has changed?
I'd say no.... but if you say no, then your project is just academic for you... and that might just be good enough for you.

[maltebp]

Sorry, for the radio-silence, but this discussion has derailed a bit beyond what is super useful thesis, so I had to down-prioritize it (not saying it's not interesting and of value to me!), and I've been super busy with work, thesis and personal-life stuff.


Super dope about the DHT documentary - I'll try and see if I can find the time to watch that. Thanks for linking that! Weird that it doesn't have more views - seem pretty cool to me.


Right, so while LibGS is not what I would want my engine to be (i.e. it's "just" software libraries, and no editor), it probably is what my engine will be for this thesis (or more than that probably). Maybe that's going to be an issue with my thesis approach, maybe not (academia is a strange world). It's definitely something I'll have to take into consideration. Thanks for bringing it to my attention.


I think I misunderstood the meaning behind Bill's quote (or at least what you meant by it). Because to me it sounds like someone trying to talk people out of using higher-level tools, but I think you mentioned it because people tends/tended to want to go lower-level than what is necessary to make a game? Regardless, I'm not a fan of the quote - to me it seem like it's overly simplifying the complicated process of game development. Because game development is not just about "making a game". It's super easy to make a game, even in assembly I'd argue. And it's not even about making a "good game". I think it's more a question about the "value" you can make in what amount of time - i.e. it's all about getting the most value per time spent. By that I don't necessarily refer to money (though, those who have a stake in the company probably do). I also mean the quality of the game - e.g. the player experience, or even the developers' experience with making it. And I think that can go both ways: you may want to move to higher level to allow more than just programmers to participate in development, or lower level in some cases, because your game's edge is a innovative technology.

Comparing Unity to Net Yaroze only really make sense when considering the historical context of the two in their respective era. Sure, back in the 90s Net Yaroze probably satisfied the indie-game dev and homebrew demand. And of course, they do have some overlap in their fundamental concepts and maybe even their runtime. But I would argue that that market has changed so significantly that trying to take Net Yaroze into the modern era and market as a "Unity for PlayStation 1" would seem a bit misleading to me. Unity is so much more than just the software APIs and the runtime. (take my opinion on this with a grain of salt, as I have very little knowledge about Net Yaroze).

And I definitely don't agree that people "just want to make crap and sell it". I agree that the some studios seem to push that agenda, but I can tell you that there are a lot of passionate game developers in the industry, who want to make the best game they can. And I think higher-level engines allow more people to bring value to game development - i.e. it's democratizing the market significantly.

I think there is a market for both small and big games - they have different appeals and different demand, but they have appeal and demand. And the barrier-of-entry for making and releasing a game has become so small that, if there is just the slightest demand, some one will make a game for it. And one of these things that have lowered this barrier are general-purpose, easily accessible game engines. It allows for smaller studios (even single people) to produce more niche experiences - which I'm personally a huge fan of.

[nocash]

I think I misunderstood the meaning behind Bill's quote

Looks so. If you still don't understand it: Just give up, you won't understand it no matter what.

(That's about what that Will or Bill was trying to say. Perhaps a bit rude, but I guess it's been meant to be a simple, clear, and reasonable advice. But now I am left clueless because I can't understand why you have projected things about values, experiences and quality into that sentence.)

[maltebp]

Looks so. If you still don't understand it: Just give up, you won't understand it no matter what.

Okay, encouraging I was just explaining why I was disagreeing with that "advice" - but that was probably just me over-analyzing it. Which I suppose makes the advice even more appropriate.

[nocash]

The quote has two things, none of it being of greater importance. There's some half-serious "winner or loser" attitude in there, which may be encouraging to some, and discouraging to other people. Apart from the harsh language, it was just saying that they did have tools that were easy to use for everyone, except those who are completely lacking the skills for game development.

Anyways, I would focus on the technical side of getting your engine going, and to see if you can get it as functional, flexible and user friendly as possible, and how far you can get within a few weeks and with or without overloading the hardware.

[maltebp]

Yeah, that's the plan, and what I've been working on as well (but still, sound advice, so thanks!). It's progressing (very) slowly, but getting somewhere at least as the game jam is approaching. Looks like it's going to be another one of those times where I've been overly optimistic about what I can do in the time given.