ESP8266 Wifi Experiments

Start a work log and update it occasionally with your projects progress
danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

ESP8266 Wifi Experiments

Post by danhans42 » October 22nd, 2018, 6:36 am

Have been meaning to have a go at this for a while, just never really got around to it.

As you will probably know, Espressif make the popular ESP8266 and ESP32, the former was launched as a simple Wifi -> Serial bridge that could be controlled with a slightly modified version of the Hayes AT command set.

The ESP8266 has a RISC core running at 80MHz/160MHz and has 80Kb RAM, upto 16 GPIO's, 802.11n Wifi and many more features I am not intelligent enough to ever probably use.

The ESP8266 ESP01 module is super cheap, around £3 or cheaper at times - $1.42 on AliExpress. It runs on 3.3v, has 3.3v IO and is very very small. Anyway, I have got a simple test working ... Telnet -> ESP01 -> PSX. At the moment I have just tried this with the Yaroze CD and it works - obviously code upload wont work because the handshaking isnt involved.

ESP01 will be useless for handshaking - it lacks enough GPIO to achieve it. An ESP12 module might be better suited for that as it breaks out more of the pins. Still, the ESP01 has RX/TX and two GPIO's to play with.

Basically, you load the ESP01 module with the Jeelabs ESPLINK firmware - this firmware is primarily designed for flashing Arduino's over Wifi - but in this case I am going to use it to pass through serial data transparently to the Playastation. I wont cover the programming of the ESP - that has been covered on the link below (just need a USB UART)

https://github.com/jeelabs/esp-link
pic2.png
So I have connected TX/RX and GND to the Playstation SIO. Once you have ESPlink loaded, you can go to the configuration web page and set the baudrate. I then ran the Yaroze boot CD
screenshot.PNG
Next steps - see what can be achieved.. it might be a simple telnet to serial gateway can be coded on the Arduino platform - as I am unsure if the ESPLINK firmware can pass the serial data through without any extra data.

Wireless PSXSERIAL would be seriously cool.

Sorry the post is scant on detail - more to follow as this evolves.
You do not have the required permissions to view the files attached to this post.

rama3
Verified
/// PSXDEV | ELITE ///
/// PSXDEV | ELITE ///
Posts: 510
Joined: Apr 16, 2017

Post by rama3 » October 23rd, 2018, 8:49 pm

A wireless Serial bridge is definitely the way to go these days.
I hope you can make it work :)

You should upgrade your ESP to one of the common dev boards. They have the full 8266 capabilities, along with power regulation and USB bridge.
NodeMCU or Wemos D1 are nice.
The Wemos even comes in a full Arduino Uno form factor, with even better power handling and protection:
https://www.ebay.de/itm/WeMos-D1-CH340- ... 2636452483

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » October 24th, 2018, 1:36 am

I have a Wemos D1, but dont need the extra functionality so far. I only need the UART pins for a serial bridge and the power is perfect for the 3.3v power output of the serial port. I use the ESP-01 because I have 30 of them on hand and my plan was to integrate it into a small PCB that slots straight into the serial port so it is powered by the playstation serial port.

If I was going bigger I wouldnt bother with the ESP8266 at all and go straight for the ESP32 Wrover or Wroom - that gives bluetooth as well as wifi and has a small footprint.

Thanks for the interest, hopefully I can. I just need to write something PC side to send the data over a TCP connection rather than serial . and make sure every single byte gets through untouched.

rama3
Verified
/// PSXDEV | ELITE ///
/// PSXDEV | ELITE ///
Posts: 510
Joined: Apr 16, 2017

Post by rama3 » October 24th, 2018, 12:18 pm

Gotta make use of those 30 boards somehow! :p
Well, I'm sure it can be done. Even if the GPIO run out, there's always multiplexer or some other trick.

Do you think a transparent Serial bridge over WiFi could upset the Serial communication?
Data can be transferred in large chunks, but there's regular long (several ms) connection stalls with WiFi.

User avatar
gwald
Verified
Net Yaroze Enthusiast
Net Yaroze Enthusiast
Posts: 282
Joined: Sep 18, 2013
I am a: programmer/DBA
PlayStation Model: Net Yaroze
Contact:

Post by gwald » October 24th, 2018, 2:00 pm

Very impressive!

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » October 24th, 2018, 5:33 pm

rama3 wrote: October 24th, 2018, 12:18 pm Gotta make use of those 30 boards somehow! :p
Well, I'm sure it can be done. Even if the GPIO run out, there's always multiplexer or some other trick.

Do you think a transparent Serial bridge over WiFi could upset the Serial communication?
Data can be transferred in large chunks, but there's regular long (several ms) connection stalls with WiFi.
Not 100% as yet in the application of the Playstation but I tested the firmware by flashing an arduino over wifi with ARVdude and it worked perfectly, so shouldnt see why not.

Dont need any GPIO for this, only the two UART pins. The ESP01 has two GPIO's that can be used, so there is a little bit of scope there. Only application that I think where GPIO would be useful would be if full handshaking was implemented - which is beyong me as it would require a firmware writing for the ESP and obviously the client stuff on the PC side.

rama3
Verified
/// PSXDEV | ELITE ///
/// PSXDEV | ELITE ///
Posts: 510
Joined: Apr 16, 2017

Post by rama3 » October 26th, 2018, 12:53 am

Ah yeah, the 8266 has a hardware UART module. I think the ESP32 does away with it, and uses software for all connectivity (since it's so flexible).

You don't happen to know if it's somehow possible to get games to output their debug printfs again? :)

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » October 26th, 2018, 6:45 pm

The ESP32 has three hardware UARTS, i think two of them can be mapped to pretty much any pins you like one you are past the bootloader. ESP32 is an amazing bit of kit - with its pin mux you can pretty much map any pin to anything within reason. Its slightly overkill for this mind you.

I have absolutely no idea on the debug info. If there is debugging information in there you might need to poke around in RAM to re enable it - but I would have thought that it would be stripped out at compile time.

rama3
Verified
/// PSXDEV | ELITE ///
/// PSXDEV | ELITE ///
Posts: 510
Joined: Apr 16, 2017

Post by rama3 » October 27th, 2018, 12:24 am

Well, thanks for telling me!
I worked with ESP32 before, using the Arduino core. I guess I never looked into it too deeply, but the Arduino documentation always mentioned how most everything is just software + that pin mux block.
The PWM capabilities are impressive as well. It's possible to configure it to generate very high frequencies, in still acceptable steps.
Using this, I created a variable clock rate SNES, with the main clock ranging from 12 to 30Mhz in pretty fine steps.
No additional hardware was required, and it controls via web ui even ;)

But back to the 8266 and WiFi Serial.
If you need any input, just ask away!
You mentioned that writing firmware would be difficult, but I think I'm doing this all the time already ;)

Ah yeah, the debug info.
True, most games have their debug prints disabled. But there's quite a lot of titles where it's still available.
I managed to get printfs from Tomb Raider 2, for example.
This was using a parallel cable and Caetla, iirc.

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » October 27th, 2018, 6:07 am

I am going to stick with the ESPlink firmware for now, as it is much more user friendly to set-up (web based interface for settings).

I suppose the first bit is to get a version of psxserial that we can pipe to a TCP port rather than serial and see what the ESPlink firmware does with the data and take it from there, which is where I am at at the moment. I am probably going to look at doing that in C or Python.

If ESPlink isnt up to the job I may well look at alternatives or writing something from scratch.

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 6th, 2019, 9:20 pm

Now back on with this after a hiatus of over a year.

I have been doing testing and can send bytes straight through the ESP8266 to the PSX at 115.2k without any loss. ESPLink does the job pretty well so cannot see the point of rewriting that - it also supports AVRdude over IP so its pretty robust. Latency could be an issue, but so long as everything gets there in the right order I cant see a problem.

The next step is writing a client for PSXserial which targets an IP rather than a hardware serial port. We aren't dealing with any flow control so can't see it being a massive task. I am working in Python initially for ease of access to libraries. I am just trying to get my head around the protocol used.

Can anyone explain in English/pseudocode how this works?

Is it just sending a sync byte? then the rest of the EXE, less the header? Any help would be greatly appreciated.

Thanks

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 8th, 2019, 2:08 am

Looking at source for the Windows client for psxserial, it looks fairly simple, I have added some annotation.

Code: Select all

 public void SendExe(string path, string com)
        {
            string text = path;                                                         // sets path to file to load
            byte[] array = null;                                                        // set up empty array for data

            SerialPort serialPort = new SerialPort(com, 115200, 0, 8);                  // setup serial port
            serialPort.StopBits = (StopBits)Enum.Parse(typeof(StopBits), "1");

            serialPort.ReadTimeout = 500;
            serialPort.WriteTimeout = 500;
            serialPort.RtsEnable = true;
            try
            {

                array = File.ReadAllBytes(text);                                        // load psx-exe into the array
                serialPort.Open();                                                      // open serial port.
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                return;
            }
            if (array.Length % 2048 > 0)  //                                            // check length of array (is it wholly divisible by 2048)
            {
                UpdateStatus("Not a valid Playstation executable !");
            }
            else
            {
                int num = array.Length / 2048;                                          // chunk data - splut into 2048 blocks                                   
                Thread.Sleep(500);
                bool flag = false;
                while (!flag)
                {
                    serialPort.Write(new byte[]                                         //write data
                    {
                        99                                                              //send sync byte
                    }, 0, 1);
                    Thread.Sleep(2048);
                    if (serialPort.BytesToRead != 0)                                    // make sure bytes to read is not zero
                    {
                        int num2 = serialPort.ReadByte();                               // reads serial port
                        if (num2 != 152)                                                // wait for EOF bye (152)
                        {
                            UpdateStatus("Sucess");
                            flag = true;
                        }
                    }
                }

                serialPort.Write(array, 0, 2048);                                       // send header
                serialPort.Write(array, 16, 4);                                         // send the rest
                serialPort.Write(array, 24, 4);                                         // send the rest
                serialPort.Write(array, 28, 4);

                SetMax(num - 1);
                for (int i = 1; i < num; i++)
                {
                    serialPort.Write(array, 2048 * i, 2048);                            //send the rest
                    UpdateStatus("Sending packet " + i + " of " + (num-1));             // prints current packet out
                    SetCurrent(i);                                                      // sets current packet
                }
                UpdateStatus("Executing the PS-EXE");
                for (int i = 0; i < 2048; i++)
                {
                    serialPort.Write(new byte[]                                         // execute 
                    {
                        255
                    }, 0, 1);
                }
                while (serialPort.BytesToWrite > 0)                                     // waits until all transfers are complete
                {
                    Thread.Sleep(100);
                }
                serialPort.Close();                                                     // close current
                UpdateStatus("Sucess");                         
            }
            SetCurrent(0);                                                          // sets current packet to zero
            SetMax(100);                                                            // maxiumum number of packets to 100
Next step, get that into python working with serial and then add functions for sending over TCP.

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 12th, 2019, 6:55 am

Started work on the client.. promising start. Currenly written in Python3 so will run on Windows/Linux easily. Only using standard modules so setting up should be fairly trivial.

[BBvideo=560,315]https://www.youtube.com/watch?v=D2vAaM9pdu0[/BBvideo]
Last edited by danhans42 on March 13th, 2019, 12:39 am, edited 1 time in total.

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 12th, 2019, 11:56 pm

Further to the above. Went trying to log serial activity to work out the protocol, and remembered that someone on here documented it.. of course, Shendo did this years ago...

Code: Select all

@rsoft: Yup. As I said I used SerialMon to capture data and analyzed it. The protocol is basically this:

1. Send sync byte (0x63) (Start of transmission).
2. Send 2048 byte EXE header.
3. Send 4 byte initial PC (X address).
4. Send 4 byte write address.
5. Send 4 byte file size.
6. Send the rest of the data.
7. Send 2048 bytes of 0xFF. (End of transmission).
Looks like I got the EOF wrong - I was only sending 0xFF once -so will make some more changes to the code later. Then datalogging to make sure that the ESP8266 and a usual USB UART are sending the same data and work from there.

Also, may need to tweak the firmware on the ESP8266 so the buffer isn't overrun. My initial issues threw up data write exceptions and the machine would crash. I then worked on throttling the speed at which packets are sent which fixed that issue.

The example I have reads the initial PC from offset 0x16 (4bytes), write address from 0x20 (4 bytes) and the filesize from 0x24 (4 bytes). Just seems confusing since the initial read of the 2048 byte header covers those 3 values.

Tips/tricks/ideas/input/feedback all welcome.

Ta

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 13th, 2019, 6:21 am

And we have success!

Fair bit to do, polishing up the client side, speed improvement and console mode. Then going to add serial support to the code so its a dual purpose tool, as I think the only other linux client for Psxserial is the java based one. The Python code is horrible, with no checking of EXE size or error catching as such.

[BBvideo=560,315]https://www.youtube.com/watch?v=8gggfl0bxIA[/BBvideo]

Its only greentro (about 45kb), but have also tried it with drugtro which is around 700kb and that also worked fine. 100% success rate so far.

I am also going to add this into a spare PSone I have, as its so small.

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 13th, 2019, 7:03 pm

Made some more adjustments to the transmit part of the code. Now the transfer speed is around 5x faster than in the video with the ESP8266->PSX connection still at 115,200bps.

Also, need to code in the handshake... it works at the moment due to their being a 2 second delay after sending the sync byte - but psxserial actually sends back a byte to state that its is ready to receive so that needs implementing ideally.

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 14th, 2019, 9:30 pm

More speed improvements. Now transfer speed is the same as using a CP2103 USB UART, with 100% reliability. Have started on the console mode, that will be a command line option so whether or not it goes into console mode post-execution.

Also, got sick of constantly reloading the PSXSerial CD, so swapped to my other Xplorer that has Unirom. It also works perfectly with the serial loader that is present in there which is pretty cool. Unirom gives console outpiut when its running which I thought might confuse things but seems fine so far.

Also, will switch back to the age-old hitserial for a bit - the source is available for that so can see about pushing up the baud rate and scope out pushing speed further. Unless there is a way to edit the current version of PSXserial by patching the EXE.

Ive dug my old PSone out, which I will fit an ESP-01 to (just need to work out how to wire this oldcrow 4 wire modchip to a PM-41(2) motherboard.

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 18th, 2019, 9:29 am

Update

Have fitted one of the ESP8266 modules to a PSone, works very well same as a regular PSX.. as expected. Simple, just connect RX, TX, GND and 3.3v and the PSone will power the ESP8266. Be careful, if you are using a USB version for reprogramming, you will need to put a diode between the PSone and the ESP8266 3.3v to stop the module backpowering the PSone from USB. Picture below shows the start of it..

psone_wifi.jpeg

Added an extra feature though - remote reset. One of the features of the ESPLink firmware is that it can reset an Arduino using a GPIO pin, I have re-purposed this feature in my application. GPIO15 is wired into the reset circuit (see link here), so that you can reset the console with a HTTP GET request. Simple python example below.

Code: Select all

import http.client
conn = http.client.HTTPConnection("192.168.0.40")
conn.request("GET", "/console/reset")
r1 = conn.getresponse()
print(r1.status, r1.reason)
The above code resets the console.

Just configure the ESP8266 as follows in the web interface..

settings.png

need to rework the client to support this, so will add command line switches to it to support it. Then need to work the console mode so you can see printf's once your code is running.

Interestingly as well, I have tested my client with programs written in tails92's psxsdk, and also the psxserial compatible client that is built in UniROM.. and both work perfectly. So you can go full open source on linux with the SDK and this python 3 client.

Brill was good enough to send me the offset to modify psxserial so it runs at 250k. Unfortunately that did not work, but will look at recalculating the values and trying some slower custom speeds if possible.

Also, made a small hole under the CD lid, so the serial data status LED can be seen, but only slightly and the hole is not noticeable.

IMG_20190317_223149.jpg
More to come..
You do not have the required permissions to view the files attached to this post.

rama3
Verified
/// PSXDEV | ELITE ///
/// PSXDEV | ELITE ///
Posts: 510
Joined: Apr 16, 2017

Post by rama3 » March 18th, 2019, 9:50 am

Following your progress closely here :)

You mentioned using an oldcrow modchip in the PSOne.
I'm not sure if that's possible with the old code, but maybe it'll work if you use the "link wire" method.
Since you already have a microcontroller in there, maybe it's feasable to use it as the modchip in some kind of background task. The ESP32 would be great for that, since it is a dual core chip and you can spend one core on just the modchip..

Anyway, the compatibility sounds great!
I understand that a console modded this way basically has a permanent WiFi Serial Comms on it?

danhans42
BANNED
BANNED
Posts: 329
Joined: Nov 28, 2012

Post by danhans42 » March 18th, 2019, 10:36 am

Hi Rama,

Thanks for the interest - got the modchip working fine with 4 wires, didn't need the link wire (will post a pic next time I open the console up)

ESPLink is only available for the ESP8266, and I think adding modchip functionailty to the ESP-Link sources is probably way beyond me, especially on one core.

That being said, if its possible to port the modchip source to the ESP32 in the Arduino IDE, I could look at coding a simple TCP-Serial in Arduino for the ESP-32 so it would perform both the modchip and serial bridge function.

I did try this with a simple sketch, but wouldnt execute the code successfully.

I do intend to play around with the ESP32 though once I am done with this version. I have a few ESP32 boards that have SD Card and 4mb PSRAM on which could be useful for buffering etc.. so will have a go with that when I have got this finished.

Basically yes, permanent serial over TCP.. so anything you send to port 23 gets echoed straight out to the PSX serial and vice versa.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests