Here’s Why We Built Our No-Code Data Organization Tool4 min read
Table of Contents
At Vue.ai, we have been developing Machine Learning based software solutions for more than half a decade at this point. And in that time, we found that for all its novelty, developing with ML is no different from developing software of any kind. The unique problems we faced actually arose from the lack of standard practices and developer skills in the field.
How do you recruit experienced developers in an industry that’s effectively a handful of years old? And how do you possibly establish a rapid development cycle when you have to define the standards yourself?
A shared language
When trying to answer those questions, we came up against another oddity. Outside of AI, it’s virtually superfluous to note that any commercially ambitious product needs to be able to scale seamlessly. Having any link between the number of users for your product and the number of engineers needed for your product to function correctly is an unforgiving cap imposed on your company’s growth. In an ideal world, we would build a product and deliver it to an arbitrary number of domain experts – catalog managers, in this case, each of whom would be able to use it to generate value.
But in practice, we saw that any ML-based product needed at least one ML engineer to be assigned to work alongside the catalog manager. This generated many challenges, not the least of which was the necessity of creating a shared language to ensure clear communication between the two. Which, considering the fundamental unconnectedness of ML with retail and other domains, was a tough undertaking,
What we needed was a solution that would let us decouple this link. A system that would not only automate the ML lifecycle to make the lives of ML engineers easier but would go a few steps further and place all that power squarely in the hands of catalog management teams – no strings attached. We now knew exactly what we needed. The problem? Nothing like that existed…
So we built it ourselves!
Introducing: Our no-code, fully customizable data organization tool
Our new catalog data organization tool is a no-code and AI-powered solution, that lets teams journey all the way from raw data to model deployment in a single interface. If there are 10,000 data points to label, the domain expert will only need to label a few hundred points in an interactive UI, while the AI system takes care of labeling the rest. The data organization tool has an integrated taxonomy management system, a robust user and task management system, annotation tooling, and much more – fundamentally built to support teams every step of the way.
Our tool decisively puts the retailer’s team in charge, so it’s not finally about automation. It’s finally about the user.
Under the hood, uninhibited by the need for human intervention, lies a library of hundreds of ML models whose combined knowledge is used to learn from every one of the user’s interactions on screen. Additionally, our teams periodically train new models with market data so that catalog teams have tailor-made models that excel at retail ontology and get better with each set of data that is imported to the tool.
The scale needed for seamless automation of this magnitude is enabled by our internal MLOps pipeline that can serve both humans and programmatic users. In a single API call, training jobs can be launched on any public cloud, in any region, and with built-in support for distributed training across a cluster of machines so that retail teams can train hundreds of models on a single set of data and auto-tune your hyperparameters.
Our teams keep track of the generated models using an in-house own model tracking system and enable immediate deployment to a production-ready, highly scalable inference system with a single click.
We can also use retailer-specific taxonomy to generate a custom logical path for inference, allowing teams to invoke any number of models in a single API call.
With infinitely scalable infrastructure, a large library of high-quality ML models, and an interface that creates a beautiful synergy between AI and humans, our catalog organization tool has quite a lot to offer and promises to unlock the next level of productivity for a retailer’s ML workloads.
We can honestly say that building and using the tool ourselves was an immensely rewarding process, a culmination of our previous struggles, and a fitting bookend to the incomplete story of the ML lifecycle.
We can’t wait to bring our interactive data organization tool to brands across the world, we’re sure retail teams going to love it as much as we do!
ABOUT THE AUTHOR
