🧠 I built a novel triple-hybrid LLM (Mamba + Attention + 32-expert MoE) from scratch for ~$50 — Titan v1 complete, Titan v2 first cycle done, expanding dataset now

@KnackAU Ray - I probably should have gone and looked at your contributions here to see what you had done, but I failed to do this earlier for which I apologise - however, honestly wasn’t pointing at you when I was commenting generally about a lack of teamworking. I guess both you and @Mati83moni Mateusz essentially are doing fundamental experimental research - you have synergistic ideas on how to improve some basics and you do experiments to see how it works out. These are the essential building blocks for useable products and they are typically done by individuals. The biggest exception to this I can think of is CERN - but historically Faraday, Curie, Einstein, and thousands of other scientists worked this way, mostly taking other people’s ideas and building upon them by coming up with something new, but occasionally doing something completely new. Products, however, are typically engineered through team efforts - often mostly commercial and driven by a single CEO, but in recent decades also organically through team-based open-source efforts. But, if you take a look on Reddit there are literally tens (or possibly hundreds) of people who have vibe coded a memory system for AI. Some of these might gain traction, but most will fall by the wayside, because it doesn’t gain many users, or because the author loses interest. In a scientific analogy these are like new life-forms coming into being - some succeeding in surviving but not evolving, and most dying off. Now imagine what the outcome might have been if these same developers had come together to work on a single memory project as a team - we would have synergy where ideas spark off one another, we would have evolution, we would have longevity, we would have a much much much better chance of achieving critical mass / virality. Now imagine extending that concept to an entire AI ecosystem - harnesses, memory, language models, infrastructure pipelines … I would imagine my own situation is a good example of the consequences - I have a history of making significant contributions to several open source projects over the past 20 years or so - but I have not yet managed to start such a project myself. I have an idea for an open source project (synergising the ideas but not the code) from two other open source projects I have contributed to) but since getting the idea 6+ years ago I haven’t had the time to start coding it, even though my shower-time thinking has taken the ideas forward. More recently I started to think about doing AI Agentic Coding to get it off the ground, but discovered just how fragmented and the agentic coding world and AI world is. I get that it is so new and fast moving, but nevertheless the lack of use of e.g. UML as part of this and the unstructured free-text approach is astounding - not building on the wealth of academic and practical knowledge as a foundation of agentic coding is an IMO amateur vibe approach.