{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreihk6rhimz6uzaoldq7q642bssdaygihbcmrprc6ebwn6sce3jtepi",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mmupmogmvrg2"
  },
  "path": "/t/medical-image-segmentation-ai-built-from-abidjan-sam-2-gradio/176248#post_2",
  "publishedAt": "2026-05-27T23:25:06.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "Since this is a medical-domain project, I’d also recommend checking out the Hugging Science Discord",
    "SAM 2",
    "SAM 2 GitHub repository",
    "SAM 2 paper",
    "NCI Medical Imaging De-Identification Project",
    "DICOM Attribute Confidentiality Profiles",
    "TCIA De-identification Knowledge Base",
    "HF Spaces storage documentation",
    "SAM2 Image Predictor Space",
    "Gradio ImageEditor",
    "Gradio AnnotatedImage",
    "Label Studio SAM integration",
    "Using SAM2 with Label Studio for image annotation",
    "MONAI Label",
    "3D Slicer",
    "OHIF",
    "CVAT COCO format",
    "CVAT segmentation mask format",
    "BiomedParseData",
    "BiomedParse model",
    "BiomedParse GitHub",
    "MedSegDB",
    "CVPR-BiomedSegFM",
    "Project Imaging-X",
    "Medical Segmentation Decathlon",
    "MedSAM dataset list",
    "MedSAM",
    "SAM-Med2D",
    "MedSAM2",
    "Sam2Rad",
    "MONAI VISTA3D-HF",
    "TotalSegmentator",
    "nnU-Net",
    "HF Model Cards",
    "HF Dataset Cards",
    "HF Model Release Checklist",
    "Hugging Face for Health",
    "Hugging Science",
    "General-Medical-AI Project Imaging-X dataset",
    "HF gated datasets documentation"
  ],
  "textContent": "I asked an LLM to explore possible improvements a bit. Since this is a medical-domain project, I’d also recommend checking out the Hugging Science Discord:\n\n* * *\n\nThis is a promising first prototype. Making a SAM 2 + Gradio demo public on Hugging Face is a good way to invite feedback, especially for a student-built project from Abidjan with an interest in African medical imaging datasets.\n\nMy main suggestion is to sharpen the positioning. I would avoid presenting the current version as a clinical AI system or a fully automatic medical segmentation model. A more accurate and stronger framing would be:\n\n> **A SAM 2-assisted medical image annotation demo for research and education.**\n\nThat wording is safer and more technically precise. It also gives you a clearer roadmap: not “replace the clinician,” but “help researchers create, inspect, refine, and export candidate annotations.”\n\n## High-level priorities\n\nIf I were improving this project, I would focus on these five areas:\n\n  1. **Positioning and safety** — make clear this is research/education only, not clinical software.\n  2. **Interactive prompting** — replace hidden or fixed prompts with visible click / box / negative-point workflows.\n  3. **Annotation export** — move from mask metadata to standard dataset formats.\n  4. **Evaluation** — add Dice/IoU and failure cases, even on a small public sample set.\n  5. **Medical / HF ecosystem alignment** — reference MedSAM2, Sam2Rad, MedSegDB, HF4H, Hugging Science, Model Cards, Dataset Cards, and privacy-aware data governance.\n\n\n\n## 1. Reposition the project\n\nI would rename or subtitle it as something like:\n\n> **SAM 2-assisted medical image annotation demo**\n\nor:\n\n> **Research demo for prompt-based medical image mask generation**\n\nI would avoid phrases like:\n\n  * “clinical AI”\n  * “diagnostic segmentation”\n  * “upload any MRI / X-ray / scan”\n  * “automatically detects medical regions of interest”\n  * “ready for clinical annotation”\n  * “confidence score” without qualification\n\n\n\nSuggested wording:\n\n> This Space generates candidate segmentation masks for 2D public or de-identified medical image slices. It is intended for research, education, and annotation workflow exploration only. It is not for diagnosis, treatment planning, triage, measurement, or clinical decision-making.\n\nThis matters because SAM 2 is a general promptable segmentation model for images and videos. It is powerful, but it is not automatically a validated medical segmentation system. See also the SAM 2 GitHub repository and SAM 2 paper.\n\n## 2. Add a visible medical safety and privacy notice\n\nFor a public HF Space that accepts medical-looking images, I would put a short warning directly in the interface, not only in the README.\n\nSuggested UI notice:\n\n> **Research / education only. Do not use for diagnosis, treatment planning, triage, or clinical decisions. Do not upload patient-identifiable images. Use only public, synthetic, or properly de-identified images.**\n\nI would also add a storage note:\n\n> Uploaded images are processed for inference only. Do not upload patient data. If this Space stores, logs, caches, or persists files, that behavior must be explicitly documented.\n\nMedical image privacy is not only about filenames. Patient-identifying information can appear in:\n\n  * DICOM metadata\n  * private DICOM tags\n  * free-text fields\n  * burned-in pixel annotations\n  * ultrasound overlays\n  * screenshots or exported PNG/JPEG images\n  * filenames and logs\n\n\n\nUseful references:\n\n  * NCI Medical Imaging De-Identification Project\n  * DICOM Attribute Confidentiality Profiles\n  * TCIA De-identification Knowledge Base\n  * HF Spaces storage documentation\n\n\n\nEven if your Space currently uses only temporary files, it is still better to say so explicitly.\n\n## 3. Replace fixed internal prompts with interactive prompts\n\nThe most important technical upgrade is the prompt interface.\n\nIf the app currently uses fixed or heuristic points internally, then it is not really segmenting the user’s intended medical target. It is asking SAM 2 to segment whatever those internal points happen to indicate. That can work for a demo, but it is fragile and hard to evaluate.\n\nSAM-style models are strongest when the target is specified interactively. I would expose the prompt controls:\n\n  * positive point: “include this”\n  * negative point: “exclude this”\n  * bounding box: “segment the object inside this box”\n  * optional rough mask or scribble\n  * multiple candidate masks\n  * mask score per candidate\n  * choose best mask\n  * refine with more prompts\n  * export selected mask\n\n\n\nA better workflow would be:\n\n  1. Upload a 2D public/de-identified image.\n  2. Select target with a click or box.\n  3. Generate candidate masks.\n  4. Add negative points if needed.\n  5. Choose or refine the best mask.\n  6. Assign a label.\n  7. Export standard annotation files.\n\n\n\nUseful UI references:\n\n  * SAM2 Image Predictor Space\n  * Gradio ImageEditor\n  * Gradio AnnotatedImage\n  * Label Studio SAM integration\n  * Using SAM2 with Label Studio for image annotation\n\n\n\nFor the current project, the key change is:\n\n> **hidden fixed prompts → visible human-in-the-loop prompts**\n\nThat one change would make the app much more credible.\n\n## 4. Be precise about supported input formats\n\nIf the app accepts `PIL` / RGB images through Gradio, then it is a 2D image-slice demo, not a DICOM/MRI/CT-volume application.\n\nI would state:\n\n  * Supported now: PNG/JPEG 2D images or exported slices.\n  * Not yet supported: DICOM, NIfTI, 3D CT/MRI volumes, DICOM SEG, PACS, clinical viewers.\n  * Not validated for: diagnosis, measurement, treatment planning, or automated reporting.\n\n\n\nSuggested wording:\n\n> This demo currently works on 2D image files. It does not yet preserve DICOM metadata, voxel spacing, orientation, slice thickness, series context, or 3D anatomy.\n\nThis makes the project look more serious, not less. Medical imaging users will trust a tool more when its boundaries are clear.\n\nFuture 3D directions:\n\n  * DICOM / NIfTI loader\n  * voxel spacing handling\n  * axial / coronal / sagittal viewing\n  * slice propagation\n  * 3D labelmap export\n  * NIfTI mask export\n  * DICOM SEG as an advanced target\n  * integration with MONAI Label, 3D Slicer, or OHIF\n\n\n\n## 5. Make the export genuinely annotation-friendly\n\nThe current JSON sounds useful, but I would not call it “training-dataset ready” unless it supports standard formats. A JSON with bounding box, area, coverage, and score is better described as **mask metadata**.\n\nA stronger export bundle would include:\n\n  * `mask.png` — binary mask\n  * `overlay.png` — visual preview\n  * `annotation.json` — internal metadata\n  * `coco.json` — COCO instance segmentation\n  * RLE mask\n  * label name / class name\n  * model name and checkpoint\n  * prompt type: point / box / mask / scribble\n  * prompt coordinates\n  * candidate mask score\n  * human-reviewed flag\n  * correction history\n  * image dimensions and preprocessing notes\n\n\n\nFor annotation-tool compatibility:\n\n  * CVAT COCO format\n  * CVAT segmentation mask format\n  * Label Studio SAM integration\n\n\n\nFor future 3D medical imaging:\n\n  * NIfTI labelmap\n  * voxel spacing\n  * orientation\n  * source volume ID after de-identification\n  * DICOM SEG, if you move toward research imaging interoperability\n\n\n\nI would phrase it like this:\n\n> The current JSON is useful mask metadata. To make it training-dataset friendly, add PNG masks, COCO JSON, RLE, CVAT-compatible export, and Label Studio prediction JSON.\n\n## 6. Add semantic labels, not only masks\n\nA segmentation mask alone is not enough for most medical datasets. You also need to know what the mask represents.\n\nFor example:\n\n  * lung\n  * liver\n  * kidney\n  * tumor\n  * lesion\n  * bone\n  * vessel\n  * polyp\n  * optic disc\n  * cell nucleus\n  * instrument\n  * background / artifact\n\n\n\nThis matters because “mask 1” is not a reusable medical annotation. A training dataset needs a label schema.\n\nRelevant resources:\n\n  * BiomedParseData\n  * BiomedParse model\n  * BiomedParse GitHub\n\n\n\nBiomedParse is interesting because it frames biomedical image understanding as segmentation + detection + recognition across multiple modalities. That is a useful direction for this project: not only “where is the mask?” but also “what is the mask?”\n\n## 7. Add a small evaluation page\n\nA public demo becomes much stronger if it includes even a small benchmark.\n\nMinimum useful metrics:\n\n  * Dice\n  * IoU / Jaccard\n  * 95% Hausdorff distance, where appropriate\n  * Normalized Surface Dice, for 3D later\n  * click count vs Dice\n  * point prompt vs box prompt\n  * SAM 2 vs MedSAM / SAM-Med2D / MedSAM2\n  * modality-wise results\n  * failure examples\n\n\n\nImportant failure cases to show:\n\n  * low-contrast boundaries\n  * small lesions\n  * noisy ultrasound\n  * overlapping anatomy\n  * multiple similar structures\n  * cropped anatomy\n  * burned-in text\n  * non-medical images\n  * poor-quality screenshots\n\n\n\nUseful datasets / benchmark references:\n\n  * MedSegDB\n  * CVPR-BiomedSegFM\n  * Project Imaging-X\n  * Medical Segmentation Decathlon\n  * MedSAM dataset list\n\n\n\nSuccess cases are good for a demo. Failure cases are what make the project useful for research.\n\n## 8. Consider a model selector, but do not start there\n\nIt may be tempting to immediately replace SAM 2 with a more medical model. I would not make that the first step. First fix the prompt UI, safety language, and export.\n\nAfter that, a model selector would be useful:\n\nModel / resource | Why it matters\n---|---\nSAM 2 | general promptable segmentation baseline\nMedSAM | medical adaptation of SAM, strong reference point\nSAM-Med2D | 2D medical segmentation adaptation\nMedSAM2 | SAM 2.1 adapted/fine-tuned for 3D medical images and videos\nSam2Rad | useful example of SAM/SAM2 prompt-learning for ultrasound\nMONAI VISTA3D-HF | 3D medical segmentation foundation model on HF\nTotalSegmentator | strong CT/MR anatomy segmentation baseline\nnnU-Net | essential task-specific medical segmentation baseline\n\nFor the current Space, I would start with:\n\n  1. SAM 2 baseline\n  2. MedSAM or SAM-Med2D for 2D medical images\n  3. MedSAM2 as a future 3D/video direction\n\n\n\nSam2Rad is especially useful conceptually because it separates autonomous, semi-autonomous human-in-the-loop, and manual prompting modes. That distinction would also be valuable in your UI.\n\n## 9. Use Hugging Face documentation patterns\n\nIf you publish a model, dataset, or improved Space, I would use proper HF documentation patterns.\n\nReferences:\n\n  * HF Model Cards\n  * HF Dataset Cards\n  * HF Model Release Checklist\n  * Hugging Face for Health\n  * Hugging Science\n\n\n\nFor a medical model card, include:\n\n  * intended use\n  * out-of-scope use\n  * supported inputs\n  * unsupported inputs\n  * training data\n  * evaluation data\n  * known limitations\n  * bias / representativeness\n  * privacy statement\n  * clinical-use disclaimer\n  * license\n  * citation\n  * hardware requirements\n  * failure cases\n\n\n\nFor a medical dataset card, include:\n\n  * source\n  * modality\n  * anatomy\n  * annotation type\n  * label schema\n  * annotator expertise\n  * de-identification process\n  * consent / ethics review, if applicable\n  * license and redistribution limits\n  * demographic/geographic coverage, if ethically shareable\n  * scanner/site metadata, if allowed\n  * known biases and gaps\n\n\n\nHF4H and Hugging Science are useful communities to look at because this is not only a computer vision project. It touches health, open science, documentation, evaluation, and data governance.\n\n## 10. Treat the Africa / Abidjan angle as a strength, but add governance\n\nThe Abidjan / Côte d’Ivoire / Africa angle is valuable. Medical AI needs more geographic diversity, more local participation, and more datasets that are not only from a few wealthy institutions.\n\nBut if the long-term goal is African medical imaging datasets, the next step is not just model engineering. It is also governance.\n\nImportant items:\n\n  * local clinical collaborators\n  * ethics approval, when needed\n  * consent pathway, when needed\n  * de-identification workflow\n  * annotation protocol\n  * annotator expertise\n  * label definitions\n  * modality and scanner metadata\n  * license and redistribution policy\n  * dataset card\n  * access control, if needed\n  * bias and representativeness statement\n\n\n\nUseful references:\n\n  * Hugging Face for Health\n  * Project Imaging-X\n  * General-Medical-AI Project Imaging-X dataset\n  * HF gated datasets documentation\n\n\n\nA good long-term vision might be:\n\n> Build an open, well-documented, privacy-aware African medical imaging annotation workflow, starting with public/de-identified samples and human-in-the-loop segmentation.\n\nThat is much more compelling than simply “SAM 2 for medical images.”\n\n## 11. Suggested roadmap\n\n### Phase 1 — Make the current demo safe and precise\n\n  * Rename as a research annotation demo.\n  * Add “not for clinical use.”\n  * Add “do not upload patient-identifiable data.”\n  * Clarify whether images are stored.\n  * Clarify supported input: 2D PNG/JPEG only.\n  * Rename “confidence” to “SAM mask score” or “model mask score.”\n\n\n\n### Phase 2 — Add human-in-the-loop prompts\n\n  * positive clicks\n  * negative clicks\n  * bounding boxes\n  * candidate masks\n  * mask selection\n  * prompt history\n  * simple correction workflow\n\n\n\n### Phase 3 — Add useful exports\n\n  * binary mask PNG\n  * overlay PNG\n  * COCO JSON\n  * RLE\n  * CVAT-compatible mask\n  * Label Studio prediction JSON\n  * label schema\n  * model/prompt metadata\n\n\n\n### Phase 4 — Add evaluation\n\n  * small public sample set\n  * Dice / IoU\n  * click-count curves\n  * modality-wise results\n  * success and failure gallery\n  * SAM 2 vs MedSAM / SAM-Med2D / MedSAM2 comparison\n\n\n\n### Phase 5 — Explore medical foundation models\n\n  * MedSAM for 2D medical prompting\n  * SAM-Med2D for 2D medical images\n  * MedSAM2 for 3D/video medical segmentation\n  * VISTA3D for MONAI-style 3D workflows\n  * TotalSegmentator / nnU-Net as practical baselines\n\n\n\n### Phase 6 — Build toward a real dataset collaboration\n\n  * dataset card\n  * de-identification protocol\n  * governance plan\n  * local clinical collaboration\n  * annotation guidelines\n  * review workflow\n  * license / access policy\n\n\n\n## 12. Concrete wording changes\n\nI would replace:\n\n> Medical Image Segmentation AI\n\nwith:\n\n> SAM 2-assisted medical image annotation demo\n\nI would replace:\n\n> Upload any X-ray, MRI or scan\n\nwith:\n\n> Upload a 2D public or de-identified medical image slice, such as PNG/JPEG exported from a public dataset\n\nI would replace:\n\n> automatically segments regions of interest\n\nwith:\n\n> generates candidate segmentation masks from user prompts\n\nI would replace:\n\n> confidence score\n\nwith:\n\n> SAM mask score, not a clinical confidence score\n\nI would replace:\n\n> JSON annotations ready for AI training datasets\n\nwith:\n\n> exports mask metadata now; PNG mask, COCO JSON, RLE, CVAT, and Label Studio export would make it more dataset-friendly\n\n## Overall\n\nThis is a good starting point. The strongest next step is not necessarily “use a bigger model.” The strongest next step is to make the project safer, more precise, and more useful as an annotation workflow.\n\nThe core transformation would be:\n\n> **from hidden fixed prompts to visible interactive prompts**\n>  **from demo metadata to standard annotation exports**\n>  **from medical-sounding claims to research/education positioning**\n>  **from isolated examples to evaluated public samples and failure cases**\n>  **from a segmentation demo to a documented dataset-collaboration workflow**\n\nThat would make the project much more useful for medical imaging researchers and more credible for open-science collaboration.",
  "title": "Medical Image Segmentation AI built from Abidjan 🇨🇮 - SAM 2 + Gradio"
}