General

1. Why do I have to provide so much information? Why can't the program just extract the foreground object?
   A computer cannot know what your definition of foreground is in every special case. The computer has no context information and does not understand the semantics of the image.
   
   
2. Will there ever be a perfect segmentation algorithm?
   We think: No. The problem is that the notion of foreground is fuzzy. There are many pictures that human beings are not able to segmentate or where it is just impossible to define any kind of unique segmentation.
   
   
3. So what is the definition of foreground then?
   We define foreground to be a set of spatially connected objects that are of interest to the user. The rest of the image is considered background. The user has to specify at least a superset of the foreground. In the GIMP 2.2 plugin or in GIMP 2.3.3 the set of foreground objects has cardinality 1.
   
   
4. Does the approach work with all images?
   There is a simple answer to this question: No.
   
   
   
5. How do you pronounce SIOX?
   Like: "sigh"-"oks". There is no intended connection to any native american people. Btw.: The Sioux are pronounced like "soo".
   
   
   
6. What are the relationships between SIOX in GIMP, the GIMP Plugin, SIOX in E-Chalk, the Java API, and the Demo-Applet?
   
   • The GIMP Plugin is a proof-of-concept implementation of an early version of SIOX. It was originally created to demonstrate SIOX to the GIMP community. It is not intended for production use.
   • The Java-API provides the reference implementation of the original idea behind the algorithm aiming towards conceptual clearness more than towards optimization in speed.
   • The demo Applet is a simple user front-end to that API, not intended for production use.
   • SIOX in E-Chalk is a research implementation running in the laboratories of the computer science department at Freie Universität Berlin.
   • SIOX is integrated into GIMP since version 2.3.3. GIMP 2.3.x is a development branch. However, there are binaries for Windows and Linux available from different sources. If you want to use SIOX in a productive environment: Please use the version integrated in GIMP 2.3.x.
   
   
   

Particular Images

1. I tried your plugin/the GIMP tool on an image and it just did not work out!
   Segmentation is a difficult problem - there is no perfect method. Please read the subsequent questions.
   
   
2. Does the algorithm work with grayscale images?
   Grayscale images contain less information than color images. The results will be worse. For plugin users: Please remember to convert your image to RGB or ARGB before invoking the plugin.
   
   
3. Does the algorithm work with indexed images such as GIF?
   Yes, but due to the color quantization the results will be suboptimal. For plugin users: Please convert your image to RGB or ARGB before invoking the plugin.
   
   
4. I tried to convert an image that contains a night/underwater/space scene and it did not work out?
   At this time, the algorithm assumes a normal daylight scene. If the illumination differs significantly from that assumption there is a high chance that the method fails. See the details page for further information.
   
   
5. What about the edges?
   In some images, the edges have to be segmentated on a subpixel basis because there is no sharp borderline. This problem also occurs if you segmentate the same image manually. At the current state of implementation of SIOX, please use feathering to blur the edges when you copy and paste an image that contains edges that are not well-defined. SIOX will provide a solution for alpha-blending in the next version. See this page for details.
   
   
6. Will there be a problem if I want to extract an object that is one of many similar foreground objects?
   Unfortunately, but there is a simple workaround: If a picture contains several very similar foreground objects, please include all in your region of interest. Using the foreground samples you can specify which object to extract.
   
   
7. How can I extract more than one object?
   For plugin and GIMP 2.3.3 users: You can only extract one spatially connected object at a time. If a picture contains several very similar foreground objects, please include all in your region of interest. Otherwise the method will fail (see above). Using the foreground samples you can specify which object to extract. Just repeat the process with all the objects.
   For GIMP 2.3.4 and higher users: Include all the objects you want to extract in your selection and use the add foreground brush as usual on a subset of the foreground objects.
   
   
8. So what kind of images WORK WELL?
   This is difficult to tell. The following factors increase the chance for a good result:
   • High color variance. If the color spectrum is very narrow, it is hard to draw distinctions.
   • Good contrast. If the object boundaries are unclear, segmentation cannot be exact.
   • The foreground object distinguishes well from the background. If you try to extract one object inside a mess of thousand that all look the same, the method will fail.
   • Good foreground and background selections. Make sure the entire object is inside your region of interest and your foreground samples are representative and do not contain any background. Please note: You cannot provide too many foreground samples.
   • The image fits to the basic assumptions. Make sure, that your notion of foreground fits our definition (see above). At this time, SIOX cannot process grayscale images or images with special lightning conditions well.
   As a rule of thumb: If the picture was taken to show a particular object - like a portrait foto of a human being, of an animal, or of a plant - SIOX will be able to extract it.
   
   
9. What do you mean by representative foreground samples?
   The foreground samples you choose should ideally contain all the colors that the foreground object contains. It is clear, that finding such a set of samples is often impossible. Use the following rules of thumb for animals or persons:
   • For animals: Select at least the face, a large part of the body, and every specialty that a specific animal might have.
   • For persons: Select a part of the face, the hair, and a part of each of the cloth he or she wears. Skin is difficult to segmentate, so select as much skin as possible.
   
   
   

Plugin Implementation

1. What does the mysterious preview button do?
   Nothing. Please ignore it. It is a technical problem with the GIMP plugin API.
   
   
2. Why does this mysterious menu appear when I do a right click on the image?
   Please ignore it. It is a technical problem with the GIMP plugin API.
   
   
3. Why can I only select one foreground rectangle?
   This will be changed when SIOX is available as a GIMP tool.
   
   
4. Why can I only do rectangular selections?
   This will be changed when SIOX is available as a GIMP tool.
   
   
5. Why is there no preview mode?
   This will be changed when SIOX is available as a GIMP tool.
   
   
6. Why can't I define sure background inside a region of interest?
   This will be changed when SIOX is available as a GIMP tool.
   
   
7. Why is the progress bar not linear?
   This will be changed when SIOX is available as a GIMP tool.
   
   
8. I want to zoom!
   This will be changed when SIOX is available as a GIMP tool.
   
   
9. I do not want to cut out - I just want a selection!
   This will be changed when SIOX is available as a GIMP tool.
   
   
10. So when does GIMP 2.4 come out?
    Do not ask us. Ask the GIMP community.
    
    
11. Is there already a preview on SIOX as a tool in GIMP?
    Yes. Please click here.
    
    
12. Can I try SIOX as a tool in GIMP?
    Yes. Please download the latest development package of GIMP (at least 2.3.3) or get the CVS devlopment tree.
    
    
13. Can SIOX be combined with other tools?
    Yes. This is recommended for tricky images. For example, try to use the freehand selection or the fuzzy selection before and/or after invoking the SIOX plugin.
    
    
14. What is the best tool to manually re-edit the selection?
    Use the erazor in both, erasing and anti-erasing mode.
    
    
15. Why do some picture take longer to process than others?
    Processing speed depends on the underlying hardware, the image resolution, and the color complexity of the image (in terms of entropy). Due to better use of GIMP's internal structures, the tool implementation in GIMP 2.4 will be faster than the plugin. At the time of writing this, the horse picture with a resolution of 1200x768 pixels took about 4 seconds to process on a 3GHz Pentium IV.
    
    
16. Will SIOX be faster when integrated into GIMP as a tool?
    Yes. There will be less overhead and GIMP's internal datastructures can be utilized better.
    
    

Video Object Extraction

1. Is the video version already available?
   Not as a standalone project. It is currently part of a yet unpublished version of the E-Chalk system.
   
   
2. Why don't you just use differential frames?
   Two reasons: First, differential frames contain both, foreground AND those parts of the background that were priorly occluded by the foreground. Second, if there is no movement, differential frames give no result.
   
   
3. Why don't you use a model of the instructor?
   Because the instructor might go away and a student might come to the board.
   
   
4. Why don't you use a model of the chalkboard content and subtract it from the video image?
   Good idea. Tell us how!