Behind the Scenes – Vue.ai’s Image Moderation Solution

Reading Time: 4 minutes

Retailers, today, are fully aware of the importance of clear, appealing product images on their sites. Internal guidelines set in place by teams also work to establish clarity and credibility to every product image on the site.

One of the biggest hurdles in ensuring this quality at scale, specifically for marketplaces, comes during the vendor onboarding process. Errors and biases, along with a slow go-to-market is the result of manually checking every single image for all products from across hundreds or thousands of sellers for guideline adherence. 

Intercepting this process with automation and AI is helping businesses improve everything from productivity to throughput to improved go-to-market time.

Vue.ai’s AI Powered Image Moderation Solution for Retailers

Vue.ai’s AI powered Image Moderation Solution for retailers has proven to work—at scale—for marketplaces in moderating enormous quantities of images from across vendors and in a nuanced, customizable way. 

Vue.ai understands that guidelines can vary across marketplaces, and sometimes even across categories. An image with a logo or multiple views in a single image might be acceptable to some, while others might reject it. Giving customer teams the tool ensures every team can custom input their requirements. They can give an ACCEPT or REJECT tag to images along with a confidence score that denotes the confidence of our models in generating the tag

For each guideline, a tag and a numeric value between 0 and 1 are provided. This number indicates the confidence of our model in providing the ACCEPT/REJECT tag. 0 represents the state of least confidence while 1 represents full confidence. This equips retailers with the ability to create their internal workflow for automatically accepting or rejecting the reviewed images based on their custom thresholds. If the confidence value of the image is greater than the set threshold, the workflow automatically accepts the images. If the value is low, moderators take a look at the image for manual verification.

The AI Behind the Solution

Convolutional Neural Network (CNN) classifiers play a critical role in the solution. The availability of their image processing functions, detectors, and segmentors makes them ideal for this use case. 

Every image submitted is evaluated to understand if it is blurry or grainy. Edge statistics from spatial filters at multiple resolutions help our models determine the clarity of every image submitted. This then determines whether or not the image matches up to the guidelines. 

A binary CNN classifier evaluates features such as watermark, logo, and price. A positive class implies the presence of these features, while a negative class implies their absence in the image. Features such as the presence of unwanted text within the image, are detected using pre-trained text detectors. These object detector models are trained to distinguish between irrelevant text over the product in the image and essential product information. Another feature common to most guidelines – specifically in retail – is the type of background in the image. Objects that are placed in plain or solid color backgrounds are preferred over distracting ones. For this, we leverage foreground-background segmentation models. Any segmented background is subject to multiple rounds of image processing before it is cleared for use.

Retailers may also sometimes require detection of the presence of children/minors in images submitted. Models such as simple binary classifiers result in a high number of false positives. At every level, we use a hierarchical solution to discard irrelevant data. This allows us to reduce the diversity in data and focus the learning on the leaf levels. 

Vue.ai image moderation solution has helped our systems process upwards of 30k requests per minute –  with zero data loss! 

Feedback & Retraining

Data in production is constantly evolving and it is critical that AI models evolve with them. This means, evaluating the submitted images and generating the relevant tags is not the final step in the workflow.

Our teams regularly sample our datasets for rejected images and analyze the AI-generated tags. The QA-based classifications are then fed back into the system and retrained to ensure that our models continue to get better over time.

Our systems are currently capable of processing standard guidelines such as borders, text, watermark, resolution, quantity of objects, presence of minors as well as custom guidelines based on individual business needs. We’ve been able to reduce the time, cost, and manual effort involved with assessing images at scale along with other benefits such as improved user engagement and productivity due to our ability to provide immediate feedback. 

Just like our data, our models are also continuously learning, evolving, and adapting to suit your needs. NSFW and offensive content, facial detection, landmarks, etc are the next steps in the pipeline.

Evaluation Criteria

Retailers looking to automate their image moderation process typically encounter two challenges:

• How to reduce the man-hours spent in manual moderation of seller data
• How to reduce the number of falsely rejected images

Vue.ai AI powered image moderation solution solves the first problem by evaluating image guideline values at scale. A comparison of our AI powered image moderation solution versus manual image qualification has shown a reduction in the time taken through our automation. In some cases, even exceeding 70%

When it comes to evaluating the second goal, it helps to keep in mind that there is an inherent data skew towards acceptable images. For this reason, the accuracy of the models isn’t an ideal measure for performance. Instead, we consider precision & recall. Our models are optimized for high precision for REJECT tags. 

With Vue.ai’s image moderation solution, retailers have seen a significant reduction in the number of inaccurately tagged images and fewer contact tickets raised by their vendors due to incorrectly tagged images.

Read More about Vue.ai’s image moderation solution