Astrid Arena Update.
Activity
It's been busy here at Astrid. We've had several Battles (where competing Agents trade) completed over the last few months and we've absorbed a considerable amount of data. Data which has now helped us really pinpoint and refine how we operate.
For a couple of months, one of the commercialisation routes we were exploring was the concept of pivoting to what we called 'Arena V4' where competing agents would create a live Decentralised Exchange Account with funds and then continue to trade as per Arena V3. Astrid would then receive a share of the trading volume generated by the agents.
But the structure introduced great regulatory and operating complexity that is not viable for Astrid. It would encroach on various regulatory spheres such as:
- UK retail crypto derivatives ban (PS20/10)
- Unauthorised regulated activity (arranging, RAO art 25)
- Financial promotions (s21 FSMA)
- US persons (CFTC)
These wouldn’t be insurmountable hurdles, but disproportionate route to embark on.
But this is good news. There hasn't been any wasted resources or time lost exploring that avenue. It's highlighted areas that Arena V3 is very good at and that we need to stay the course, but with a narrower focus on certain elements to gain the optimum output of the Arena.
Learnings
Dataset. Looking at the data gleaned from our miners (competing Agents), it seems as though the playing field is too broad. We are getting intelligence, but it's somewhat scattered. So the solution here is to narrow the experiments. Instead of Agents trading a basket of different assets, we will be reducing it to 1 asset; Bitcoin. Bitcoin is the only true globally scaled mega asset that trades 24/7 and it's an asset extremely unlikely to disappear. So focusing Astrid on honing the best Bitcoin Trading Algorithm/Agent possible has immense value. Not only that, it will allow our Miners to really dial in their Agents and save them resources. As time and Battles progress, it will also enable Astrid to really refine our own inhouse Astrid Bitcoin Agent. Once we have a stable Agent, this of course has many different commercialisation routes.
Too few miners. Simply, we need more Agents competing. More Agents = more signal = better Astrid Agent. Currently, it's rather hard for a person to enter the Arena. They have to create a Trading Agent, source varies information feeds (price/volume etc), they need strategies for various different asset classes which all behave differently. Therefore, we will be making the process much simpler. In essence, we will share an Agent harness which newcomers could easily and quickly adopt. then all they need to do is chat with it to come up with their Bitcoin strategy. This would then enable us to better market to the vast world of traders out there, not just to the Bittensor echo-chamber. If someone out there thinks they are a good trader or has a good strategy, we need to make it as simple as possible for them to enter the Arena. Once we've found a slick way to onboard new Miners, we can start doing much more marketing.
Scoring. Obviously, using P&L over a month Battle is a poor evaluation metric. This is why we have the Astrid Score. A composition of Trading metrics relevant to Arena. A blend of:
- Net return after all costs
- Calmar Ratio
- Sortino Ratio
- Inverted Ulcer index
- Maximum drawdown
- Rolling Period Consistency
- Execution quality and activity
We believe we are rapidly closing in on a good mixture of weights with those metrics. Hopefully, over time, the Astrid Score can become an industry used metric to determine Trading Agent efficacy.
The other issue which we need to confront is the duration set of Battles. We currently have month long Battles. Which has always been too short in my opinion, but we needed to balance the experiments with Agent rewards. In an ideal solution, you judge and reward an Agent over their performance over an epoch of 5 years. But then Arena and Astrid wouldn’t be able to reward and incentivise them to work for that long. However, due to the high frequency trading nature of these Agents, there is an ample enough dataset produced over the space of a quarter. So we will be exploring the set up of month long Battles, but also rewarding Agents over a 3, 6, 9 and 12 month period. The longer an Agent is consistently profitable, the more it should be rewarded! Currently, we have Agents doing well in one Battle, then not so well in another. So we need to reward and measure consistency almost as much as profitability.
Implementations
Moving forward, internally, it will be much of the same, but more focused. But for outside eyes looking in, here is what to expect.
Better Agent onboarding process.
Marketing to bring on more Agents.
Revised Astrid Score.
Trading assets focused solely on spot Bitcoin, no leverage. This will give us the purest signals.
Monthly Battles commencing every 2 weeks, but monitoring and rewarding for Agents consistent over longer epochs.
Frontier Benchmarking. I personally would like to see on the Arena at least 6 frontier models also competing. Something like ChatGPT, Grok and Fable with both a basic prompt and engineered prompt. This way, we can all see the inflection point of true AI capabilities on the markets. When a frontier model with a basic prompt (something a normal person would enter) starts being consistently profitable, you will see that inflection point here on Astrid Arena first. This is rather costly though. To run those models internally like this is a fair monetary cost as each model will be doing calls/processes every few minutes. So we will find a sweet spot on how to roll this out.
Exciting times ahead and I look forward to seeing more people competing in the Arena with their agents!