Background

We embarked on building an AI system capable of analyzing brain scans, identifying anomalies, and aiding in understanding any detected abnormalities. The challenge was not just to train the model to “see” but to view with precision and clarity, particularly when abnormalities mimic healthy tissue visually.

What We Tried (and Why)

Initially, we adopted a straightforward strategy: enhance model training with visual cues like circles around abnormalities in MRI scans to guide the model’s focus. The rationale was that explicit markers could enhance the learning process by highlighting anomalous regions.

What Broke or Didn’t Work

The approach misfired; the model learned to detect the circles rather than the anomalies themselves.

📌 Design Principle: Ensure training data annotations do not introduce artifacts that models might erroneously learn.

The Shift We Made

We revised our method by removing visual annotations from training data. Instead, we manually identified and annotated regions of interest during the training process. This change necessitated significant effort but ensured that the model relied on medical data, not artificial markers.

What Worked (and What Still Doesn’t)

With unmarked data, the model showed high detection accuracy (80%-95%) for smaller lesions. However, performance faltered (~50% accuracy) with larger, more ambiguous damage due to diffuse boundaries and complex patterns. It’s clear where the model excels and where further refinement is needed.

Tradeoffs and Strategic Decisions

Option A Option B
Focus on refining model for common, well-classified cases Develop a generalized model for all cases
Offers quick deployment and immediate value in clinical settings Future-proofs tool for edge cases; may delay completion

Open Questions We’re Still Exploring

  • How to effectively manage model switching between different types of anomalies?
  • What’s the best strategy for visual labeling to avoid interoperability issues?
  • How can we balance the tradeoffs between a general model versus specialized models?

We believe we can reframe software delivery from the ground up, where every decision, tool, and interaction is guided by contextual intelligence.

If You’re Solving Something Similar…

We’re eager to collaborate with those who’ve tackled similar tradeoffs in medical imaging or other domains to share insights and explore potential solutions.

Contact: hello@cadabra.studio
More at: https://cadabra.studio

Originally published on: