Two of the most popular open source labeling options are Diffgram and LabelStudio.
Overall Diffgram has:
- Easier UI based customization
- More extensive automations
- Similar media type coverage
- Greater depth of image features
- Greater depth of video features
- Greater depth of text features
- 3D Support (Missing in Labelstudio)
- Feature Store – Ingest, Store, Query, Stream (Missing in Labelstudio)
- Better Scale
- Overall Richness of Features
- Ease of use, with everything available on the UI
- All in one product, truly open source
Example – Label Schema Options
As you can see in this video it’s equally easy to create labels in both platforms. However, Diffgram also has support to easily create complex attributes such conditional selects, free text, multiple select, tree view, radio buttons, sliders, time, date etc. Diffgram has the ability to capture your Schema needs no matter complex.
Deep Dive Comparison
We compare the depth of interfaces and overall features.
Depth of Interfaces
A comparison of depth of features in key image, video, 3d, and text areas.
| Concept | Diffgram | LabelStudio | Winner |
| Depth of Image | AutoBorder (True Segmentation) Multi-Spatial Polygons, Tracer User Defined Spatial Types Keypoint Annotation Template Depth Copy, Share, History Snap to Annotation UI Customizable Settings Overall depth | Missing |  |
| Depth of Video | Box, Polygon, All spatial types on videos Video Segmentation Next Frame Suggestions Complex video series with previews Long/High frame rate video | Missing | |
| Depth of 3D Support | 3D Data Labeling | Not Supported | |
| Depth of Text Support | Draw relationships (no button press) Dedicated text rendering engine, labels do not reflow text Auto label all similar words Auto research word | Missing | |
Overall Product
Comparison of overall product features, including major areas like Feature Store.
| Concept | Diffgram | LabelStudio | Winner |
| Customization | UI based creation of Schema and Attributes What you See is What you Get (UI Editor) Automatically senses right UI to display | Manually Coding | |
| Automation | Interactive Userscripts, or custom backend | Custom Backend Only | |
| Media Type Coverage | Current: Image, Video, 3D, Text. Coming soon: Audio, TimeSeries, Geo, DICOM, generic document rating | Current: Image, Text, TimeseriesComing Soon: Video | Tie |
| Feature Store | Import Wizard (UI based ingestion) All-Cloud Browser (AWS, GCP, Azure) Query Labeled Data Stream data directly to pytorch/tensorflow memory | Missing | |
| Overall Richness of Features | Automatic task assignment Automatic task creation Multi-stage pipelines Export to cloud Quality Assurance Features Exams to get your new recruits up to speed. | Missing | |
| Scale | Proven scale to 100M+ annotations on single database and 10B+ annotations with Citus. Project and Tasks based organization. | Missing | |
| Overall Ease of Use | Nearly everything can be done in the UI “Wizard” based approach for all Admin activities – step by step on everything Best in class UI/UX for annotation Maintains better ease of use despite having dramatically more breadth and depth. | Missing | |
| Enterprise Support | All Features included upfront. Open source limit is 20 users. | Limits, many enterprise features are actually closed source. | |
Labelstudio’s main claim to fame was the overall coverage of interfaces. We have been working on Audio, TimeSeries, GEO/SAR, DICOM, generic document rating interfaces. As shown above if you have Image, Video, 3D, or Text then Diffgram has greater depth. If you have another need we would love to talk with you as we finish building these interfaces, please contact us, or get on the waiting list now for a specific UI.
Myths Busted
Is Label Studio is Truly Open Source? No!
Label studio includes severe restrictions on workflow and other concepts in the portion of code they have open source. Whereas Diffgram includes all features in open source.
Diffgram include all these features in open source (LabelStudio/Heartex doesn’t):
- Persistent storage
- Dataset management
- Kubernetes use (K8s) Helm Chart
- Roles and Permissions (RBAC)
- Review Workflows
- Label queue management
- Management Reports
- Data Stays on your servers
- Audit Log
- Many more…
Is Label Studio More Customizable? No!
Because label studio shows part of their UI in code, and they do a good job showcasing multiple examples, it makes it look like there’s more customization. In reality, virtually anything you can customize in Labelstudio you can do it in Diffgram, and often from the UI. Plus Diffgram allows you to customize some things that Labelstudio doesn’t, like creating user defined spatial types.
For example in this video you can see that both can be customized, however Diffgram Schema is automatically reused with no extra work.