Well I will see how well I can write this morning given the early morning slumber party going on in the den right now. Yesterday was Eldest’s birthday, and instead of a birthday party we let her BFF stay over. The two of them and Younger have been up the last two hours having a dance party listening to songs, singing along loudly and stomping around the room. It’s cute, but very distracting.
I spent last night delving into the internals of Redis, trying to figure out how to clear out some market data that got entered incorrectly due to an error on my part. I got that done while I was trying to figure out how to get historical data for the Serum event queue. It’s a circular buffer, so once it clears out it seems the only way to get the data is via the Solana explorer, feeding it into the Serum JS library to replay and decode the instructions. I think anyways. I’m waiting for confirmation, but I have no idea whether such a tool exists or whether we’d have to build one from scratch.
I went to bed reading over the Solana framework docs. I managed to skim through last night and learned a bit. I really don’t care much about most of it, but it did confirm some of my hunches about the ephemeral nature of Solana. A validator node can not be expected to store a complete copy of the Solana blockchain the way a Bitcoin or Ethereum node does. Apparently they’re going to use Google’s BigData as a storage medium for data after six months or so.
I woke up and immediately picked up my iPad and started reading the developer docs again, and took a look at the spl-token code again. It made a lot more sense this time, I guess I’ve started picking up some Rust skills after all. One thing that has finally made sense is that data in Solana programs is obscured, meaning that it has to be interpreted by the program that it is intended for. Looking at the data in the Explorer or SolScan is only going to get you so far to figure out what the hell is going on.
Coming from Python, I’ve been struggling with ways to make the NodeJS REPL work in a similar way. Import functions aren’t really allowed in the REPL, although requires are. Trying to load functions in real time and use them to debug live, on-chain data is a bit of a challenge. I made a couple of in-roads with that last night, and was able to do a bit of playing around in the serum-history module to see exactly what it was doing, but it still confirmed my hunch that Serum event data was ephemeral.
I see a lot of opportunity here if we can build a module that can recreate event data from Serum marketplaces, tracking cost basis is an obvious example. I can tell you that tax season for Solana is going to be almost impossible for people that aren’t taking good notes.