Spaces:
Running
Running
| title: MLSE Player 3D Generator | |
| emoji: 🏀 | |
| colorFrom: blue | |
| colorTo: green | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| # MLSE Player 3D Generator | |
| A 3D player model generator that uses Meta's SAM 3D Body model to convert images of athletes into detailed 3D models. | |
| ## Overview | |
| This application allows you to upload images of players and automatically generate 3D human body models. It uses Meta's SAM 3D Body technology, which is designed for high-quality 3D human mesh reconstruction from a single image. | |
| ### Features | |
| - Upload player images and generate 3D models | |
| - Automatic player detection and segmentation | |
| - High-quality 3D human mesh reconstruction | |
| - Interactive 3D viewer for examining models | |
| - Export models in GLB format for use in various applications | |
| ## Setup | |
| ### Prerequisites | |
| - Access to Hugging Face model repository | |
| - GitHub account for Hugging Face Spaces | |
| ### Deployment Options | |
| #### 1. Hugging Face Spaces (Recommended) | |
| 1. Fork this repository on GitHub | |
| 2. Create a new Space on Hugging Face Spaces (https://huggingface.co/spaces) | |
| 3. Link your GitHub repository to the Space | |
| 4. Choose "Docker" as the Space SDK | |
| 5. Configure the Space with GPU hardware (required for optimal performance) | |
| 6. Set up the Hugging Face access token as a secret named `HF_TOKEN` | |
| 7. Deploy the Space | |
| #### 2. Local Development | |
| 1. Clone this repository | |
| 2. Install dependencies: | |
| ```bash | |
| pip install -r requirements.txt | |
| # Install Detectron2 | |
| pip install 'git+https://github.com/facebookresearch/detectron2.git@a1ce2f9' --no-build-isolation --no-deps | |
| # Optional: Install MoGe | |
| pip install git+https://github.com/microsoft/MoGe.git | |
| ``` | |
| 3. Run the application: | |
| ```bash | |
| python -m app.main | |
| ``` | |
| ### Environment Variables | |
| - `HF_TOKEN`: Your Hugging Face access token (required for model download) | |
| - `PORT`: The port to run the server on (default: 7860) | |
| ## Usage | |
| ### API Endpoints | |
| #### Upload an Image | |
| ``` | |
| POST /api/upload | |
| ``` | |
| Form data: | |
| - `file`: Image file (JPEG/PNG) | |
| - `player_name`: Name for the model (default: "player") | |
| - `use_keypoints`: Whether to use keypoint detection (default: true) | |
| - `use_mask`: Whether to use segmentation masks (default: true) | |
| #### Process Base64 Image | |
| ``` | |
| POST /api/process | |
| ``` | |
| JSON body: | |
| ```json | |
| { | |
| "image_data": "base64_encoded_image_data", | |
| "player_name": "player_name", | |
| "options": { | |
| "use_keypoints": true, | |
| "use_mask": true | |
| } | |
| } | |
| ``` | |
| #### Check Job Status | |
| ``` | |
| POST /api/status | |
| ``` | |
| JSON body: | |
| ```json | |
| { | |
| "job_id": "job_id_from_upload_response" | |
| } | |
| ``` | |
| #### List All Jobs | |
| ``` | |
| GET /api/jobs | |
| ``` | |
| #### Get Model File | |
| ``` | |
| GET /api/model/{job_id} | |
| ``` | |
| ## Integration with MLSE Platform | |
| To integrate this service with the MLSE platform: | |
| 1. Deploy this service on Hugging Face Spaces | |
| 2. Add the Player 3D Generator to the MLSE AI tools section | |
| 3. Configure the frontend to call this service's API endpoints | |
| 4. Use the returned model URLs to display 3D models in the MLSE platform | |
| ## Technical Details | |
| ### Model Information | |
| This application uses SAM 3D Body, a foundation model for 3D human body reconstruction from Meta AI: | |
| - Model: `facebook/sam-3d-body-dinov3` (or `facebook/sam-3d-body-vith`) | |
| - Papers: [SAM 3D: Segment Anything in 3D with Momentum Human Rig](https://github.com/facebookresearch/sam-3d-body) | |
| ## License | |
| This project uses SAM 3D Body which is covered by Meta's license. Please check the [original repository](https://github.com/facebookresearch/sam-3d-body) for license details. |