Two of the most popular open source labeling options are Diffgram and LabelStudio. 

Overall Diffgram has:

  • Easier UI based customization
  • More extensive automations
  • Similar media type coverage
  • Greater depth of image features
  • Greater depth of video features
  • Greater depth of text features
  • 3D Support (Missing in Labelstudio)
  • Feature Store – Ingest, Store, Query, Stream (Missing in Labelstudio)
  • Better Scale
  • Overall Richness of Features
  • Ease of use, with everything available on the UI
  • All in one product, truly open source

Example – Label Schema Options

As you can see in this video it’s equally easy to create labels in both platforms. However, Diffgram also has support to easily create complex attributes such conditional selects, free text, multiple select, tree view, radio buttons, sliders, time, date etc. Diffgram has the ability to capture your Schema needs no matter complex.

Deep Dive Comparison

We compare the depth of interfaces and overall features.

Depth of Interfaces

A comparison of depth of features in key image, video, 3d, and text areas.

Depth of Image AutoBorder (True Segmentation)
Multi-Spatial Polygons, Tracer
User Defined Spatial Types
Keypoint Annotation Template Depth
Copy, Share, History
Snap to Annotation
UI Customizable Settings
Overall depth
Depth of Video Box, Polygon, All spatial types on videos
Video Segmentation
Next Frame Suggestions
Complex video series with previews
Long/High frame rate video
Depth of 3D Support3D Data LabelingNot Supported
Depth of Text SupportDraw relationships (no button press)
Dedicated text rendering engine, labels do not reflow text
Auto label all similar words
Auto research word

Overall Product

Comparison of overall product features, including major areas like Feature Store.

CustomizationUI based creation of Schema and Attributes
What you See is What you Get (UI Editor)
Automatically senses right UI to display
Manually Coding
AutomationInteractive Userscripts, or custom backendCustom Backend Only
Media Type CoverageCurrent: Image, Video, 3D, Text.
Coming soon: Audio, TimeSeries, Geo, DICOM, generic document rating
Current: Image, Text, TimeseriesComing Soon: VideoTie
Feature StoreImport Wizard (UI based ingestion)
All-Cloud Browser (AWS, GCP, Azure)
Query Labeled Data
Stream data directly to pytorch/tensorflow memory
Overall Richness of FeaturesAutomatic task assignment
Automatic task creation
Multi-stage pipelines
Export to cloud
Quality Assurance Features
Exams to get your new recruits up to speed.
ScaleProven scale to 100M+ annotations on single database and 10B+ annotations with Citus.
Project and Tasks based organization.
Overall Ease of UseNearly everything can be done in the UI

“Wizard” based approach for all Admin activities – step by step on everything

Best in class UI/UX for annotation

Maintains better ease of use despite having dramatically more breadth and depth.
Enterprise SupportAll Features included upfront. Open source limit is 20 users.
Limits, many enterprise features are actually closed source.

Labelstudio’s main claim to fame was the overall coverage of interfaces. We have been working on Audio, TimeSeries, GEO/SAR, DICOM, generic document rating interfaces. As shown above if you have Image, Video, 3D, or Text then Diffgram has greater depth. If you have another need we would love to talk with you as we finish building these interfaces, please contact us, or get on the waiting list now for a specific UI.

Myths Busted

Is Label Studio is Truly Open Source? No!

Label studio includes severe restrictions on workflow and other concepts in the portion of code they have open source. Whereas Diffgram includes all features in open source.

Diffgram include all these features in open source (LabelStudio/Heartex doesn’t):

  • Persistent storage
  • Dataset management
  • Kubernetes use (K8s)  Helm Chart
  • Roles and Permissions (RBAC)
  • Review Workflows
  • Label queue management
  • Management Reports
  • Data Stays on your servers
  • Audit Log
  • Many more… 

Is Label Studio More Customizable? No!

Because label studio shows part of their UI in code, and they do a good job showcasing multiple examples, it makes it look like there’s more customization. In reality, virtually anything you can customize in Labelstudio you can do it in Diffgram, and often from the UI.  Plus Diffgram allows you to customize some things that Labelstudio doesn’t, like creating user defined spatial types.

For example in this video you can see that both can be customized, however Diffgram Schema is automatically reused with no extra work.