mAP

[GOATmessi8]

Have you ever evaluate the transformed trained model in VOC2007? I've tried your code and got a 71.9 mAP while the original is 76.8. Then I found a tiny error in test code, after fixing the result up to 72.8 mAP, still not enough...

[longcw]

Yes, I got the same result. You can make a pull request for me to fix the bug.
I have no idea about the low mAP of my implementation. Did you try darknet implemented by the author?

[GOATmessi8]

Not yet, but I found an issue in darkflow, it seems the transfer to tensorflow also cause some difference. https://github.com/thtrieu/darkflow/issues/25
I will make a pull request if I figure out the training part. Maybe that could solve the problem...

[longcw]

I implemented the loss function following the darknet and the training process is work now.
I trained it on VOC2007 trainval set and got a 71.86 mAP ~50mAP on the test set.
Maybe you can find out some other problems about the low mAP with the help of darknet source code.

[terrychenism]

@longcw Thank you for sharing code. I have tested the converted darknet model, which got ~72 mAP. Then I trained VOC07 trainval set for 160 epoch (totally use your github codes), which only got ~50 mAP. Did you successfully train the yolo2 detector?

[longcw]

Thank you for your comment.
I tested the trained model and got the same result, ~50mAP. There are still some bugs for training. I am sorry for this.

[crazylyf]

For test phase, there are two parameters inconsistent with the original darknet:

• The thresh parameter for bbox filtering is 0.001 in darknet, while it is 0.01 in test.py;
• The iou_thresh for nms is 0.5 in darknet, while it is 0.3 in this project.
  For train phase, the thresh in cfgs/config.py should be 0.24, instead of 0.3;

As @ruinmessi , before correcting those parameters, the mAP in VOC2007-test is 71.9. Correction of first parameter improves slightly to 72.2, and correction the iou_thresh further boosts to 73.6.
The tensorflow version of yolo (darkflow) seems to suffer such a problem too, and an issue of that project pointed out some possible reasons. Maybe the reasons exist also in this project?

[crazylyf]

@ruinmessi What error in test code have you fixed?

[GOATmessi8]

@longcw @crazylyf Sorry for leaving a long time. I boost the mAP to 74.3 by changing the nms order like this while this project do the nms in a function called postprocess. with the exact parameters you mentioned.

[crazylyf]

Why your mAP is 0.7 higher if we are using the same parameters? Am I missing something?

[GOATmessi8]

The nms should implement before thresh holding.

[longcw]

@ruinmessi Thank you for pointing out this problem.

[GOATmessi8]

@longcw I am curious about how to convert the original weights to h5 file, could you please show me some details or scripts?

[longcw]

@ruinmessi I use darkflow to load original weights from the binary weights file.

[rdfong]

Is there any update on the training issue?