conflicting dependencies

[bertsky]

If I follow the installation steps currently in the README, I get the following from the last pip command (i.e. when installing layout-parser a second time, after installing detectron2):

Installing collected packages: pycocotools, fvcore
  Attempting uninstall: pycocotools
    Found existing installation: pycocotools 2.0.2
    Uninstalling pycocotools-2.0.2:
      Successfully uninstalled pycocotools-2.0.2
  Attempting uninstall: fvcore
    Found existing installation: fvcore 0.1.2.post20210128
    Uninstalling fvcore-0.1.2.post20210128:
      Successfully uninstalled fvcore-0.1.2.post20210128
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ocrd-segment 0.1.3 requires pycocotools>=2.0.2, but you have pycocotools 2.0.1 which is incompatible.
detectron2 0.3 requires fvcore<0.1.3,>=0.1.2, but you have fvcore 0.1.1.post20200623 which is incompatible.
detectron2 0.3 requires pycocotools>=2.0.2, but you have pycocotools 2.0.1 which is incompatible.
Successfully installed fvcore-0.1.1.post20200623 pycocotools-2.0.1

So it seems we are in conflict with current master of detectron2 here.

Could you point me to the right version of detectron2 to fetch, or update layout-parser accordingly?

Also, why not make the detectron2 dependency explicit in setup.py?

[bertsky]

I am not sure if this is related: When I ignore the above errors and continue with detection, I see these:

>>> import layoutparser as lp
** fvcore version of PathManager will be deprecated soon. **
** Please migrate to the version in iopath repo. **
https://github.com/facebookresearch/iopath

>>> layout = model.detect(image)
/data/ocr-d/ocrd_all/venv/lib/python3.7/site-packages/detectron2/modeling/roi_heads/fast_rcnn.py:154: UserWarning: This overload of nonzero is deprecated:
	nonzero()
Consider using one of the following signatures instead:
	nonzero(*, bool as_tuple) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  filter_inds = filter_mask.nonzero()

And layout is empty.

[bertsky]

Tried with the last stable release, i.e. detectron2==0.3, but still the same result.

[bertsky]

0.2.1 (on torch==0.1.6) is also the same.

Note, one cannot just ignore detectron2's fvcore requirement, since it is enforced:

assert get_version(fvcore, 3) >= (0, 1, 2), "Requires fvcore>=0.1.2"
AssertionError: Requires fvcore>=0.1.2

[bertsky]

And layout is empty.

That might be a specific problem of the lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config model. I do get non-empty results for all the other models (including ResNet-50 based PubLayNet models) – but the results are pretty much useless for my historical documents.

I can provide samples from layoutparser.draw_box().

[lolipopshock]

Thanks for reporting this issue! There has been some updates for Detection, and the support for fvcore is somewhat problematic in the current version. I plan to remove the fvcore dependency in the next major release, which should be around within 1 month.

And speaking of the models

1. it generates no outputs because it does not detect any boxes more than the specified confidence thresholds. You could try adding the extra_config when initializing the models to lower the threshold and try to see some outputs:

   lp.Detectron2LayoutModel(
               config_path = xxx,
               extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.25], # reduce the values)

2. I think your dataset looks significantly different from the PubLayNet dataset and the others. If possible, you could share a screenshot of your doc and I could better help you identify the best models. But ultimately, you might need to create the annotations yourself and train you own models. Speaking of that:
   • for the pre-training issue you mentioned earlier, it should be fixed within 1wk
   • and we are working on an annotation tool for more efficient layout annotation. it should be released within 1mo, and please stay tuned.

[bertsky]

Many thanks for your fast response!

Good to know you are already onto these dependency conflicts.

So am I correct in inferring these warnings don't actually do any harm?

1. it generates no outputs because it does not detect any boxes more than the specified confidence thresholds. You could try adding the extra_config when initializing the models to lower the threshold and try to see some outputs:

   lp.Detectron2LayoutModel(
               config_path = xxx,
               extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.25], # reduce the values)

Understood. I went as far down as lp.Detectron2LayoutModel('lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config', extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.01]), but still got no results there.

2. I think your dataset looks significantly different from the PubLayNet dataset and the others. If possible, you could share a screenshot of your doc and I could better help you identify the best models.

With pleasure:

• original:
  
• on newspapernavigator Mask-RCNN model:
  
• on primalayout Mask-RCNN model:
  
• on publaynet (small) Mask-RCNN model:
  

But ultimately, you might need to create the annotations yourself and train you own models. Speaking of that:

• for the pre-training issue you mentioned earlier, it should be fixed within 1wk
• and we are working on an annotation tool for more efficient layout annotation. it should be released within 1mo, and please stay tuned.

Wonderful!

[Jim-Salmons]

@bertsky, nice to virtually meet you. Found your Leipzig institutional affiliation via search engine sleuthing. :-) So wish you were on Twitter so you could be engaged in the fine-grained research chatter related to the intersection of Digital Humanities and AI/ML wrt digitization pipelines. As an independent Citizen Scientist, that communication channel is invaluable to me.

You must be buddies with Clemens Neudecker and the OCR-D network given your recent involvement in that mega project. I know them through involvement in the Time Machine EU project, NewsEye, DATeCH, and Andreas Maier’s Pattern Recognition lab at FAU etc. First noticed your GitHub comment about LAREX’s PAGEgts reading order issue. I resonated with that as magazines and newspapers have complex document structures where within page reading order is indeterminate while being determinant at the issue level.

It is not surprising to see you here kicking the tires of Zejiang @lolipopshock Layout-Parser. The PDF annotation tool Zejiang alludes to is #PAWLS from @allenai lab here: https://github.com/allenai/pawls. Looks awesome and could be very helpful to my #MAGAZINEgts research. Would corresponding to you via your Leipzig email address be okay? Or do you have a preferred way to chat about shared interests?

[Jim-Salmons]

BTW @bertsky, given my OCDish bent, I actually spent a looonnnggg day getting Layout-Parser installed and working on my Windows 10 development box. I told Zejiang @lolipopshock I’d contribute an installation note to that effect once I confirmed the procedure. So far I have not been successful rewalking that torturous process. :-/ Will likely wait for a Docker release so as not to need to knock myself out with future updates. :-)

[lolipopshock]

Many thanks for your fast response!

Good to know you are already onto these dependency conflicts.

So am I correct in inferring these warnings don't actually do any harm?

1. it generates no outputs because it does not detect any boxes more than the specified confidence thresholds. You could try adding the extra_config when initializing the models to lower the threshold and try to see some outputs:

   lp.Detectron2LayoutModel(
               config_path = xxx,
               extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.25], # reduce the values)

Understood. I went as far down as lp.Detectron2LayoutModel('lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config', extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.01]), but still got no results there.

1. I think your dataset looks significantly different from the PubLayNet dataset and the others. If possible, you could share a screenshot of your doc and I could better help you identify the best models.

With pleasure:

• original:
  
• on newspapernavigator Mask-RCNN model:
  
• on primalayout Mask-RCNN model:
  
• on publaynet (small) Mask-RCNN model:
  

But ultimately, you might need to create the annotations yourself and train you own models. Speaking of that:

• for the pre-training issue you mentioned earlier, it should be fixed within 1wk
• and we are working on an annotation tool for more efficient layout annotation. it should be released within 1mo, and please stay tuned.

Wonderful!

Yes, please ignore the warnings. And the document is significantly different from the training datasets, so it's expected to generate the non-sense outputs.

[bertsky]

@Jim-Salmons, my pleasure!

Regarding communication channels, Twitter's not my cup of tea TBH, but you can reach me under the same nick at gitter.im. Probably not a replacement for public fine-grained research chatter, though. But as an open forum frequented by researchers and practitioners in the field alike, I can recommend and invite you to https://gitter.im/OCR-D/Lobby. (And yes, my Uni Leipzig email is also fine of course.)

Yes, I do know Clemens, and indeed continue to work on that mega-project (on and off, also hoping to stay involved in the next round).

I agree, reading order is especially tough on newspapers and magazines. Incidentally, I have recently come to use a nice new layout tool (a contribution from SBB to the Qurator project co-chaired by Clemens IIRC) which IMHO excels in that respect: eynollah.

PAWLS, I've heard of it, and welcome seeing a fresh new approach to GT production/annotation, with different focus than LAREX, Transkribus etc. But such tools only become interesing to me as soon as they support ALTO or PAGE.

But I must confess I don't know anything about your MAGAZINEgts yet, so I've got to catch up with reading in that area I guess...

[bertsky]

@lolipopshock

And the document is significantly different from the training datasets, so it's expected to generate the non-sense outputs.

I see. I had my hopes up a little for the HJDataset model, because they are equally historic and are similar in visual appearance. But of course, vertical text in horizontal columns and horizontal text in vertical columns are still very different kinds of layouts. Anyway, for the sake of completeness, here's the result of applying that model to my sample after 90° reflection:

So I guess I should really start training my own models now. But I am wondering: Do you have any experience how well mixing different datasets (in the hope to increase generality) or transferring weights from a larger model like the full PubLayNet to a small domain model works?

[bertsky]

I noticed that I fed images in BGR instead of RGB (the cv2.imread example in the README does not explain this, should have read the other documentation first). But it does not make a difference. (Also, strangely, all models have input.format: BGR in their configuration...)

Then I tried to do binarization, deskewing and cropping externally. This did help somewhat:

• on primalayout Mask-RCNN model:
• on publaynet (full) Mask-RCNN model (non-empty now):

I wonder what the masks would look like. Is there a way to query / plot them?

Also, getting back to vertical text and in-domain data, here's a different set of examples:

original	on HJDataset model

Is that expected output, too? Why do I get the feeling I am making some user-side mistake?

[bertsky]

(In fairness, I do get good looking results on PubLayNet itself. Here are 3 images from the validation subset:

original image	on publaynet (large) model

Looking forward to your expert opinion on my above question how this could become a base model for other domains...

[ieriii]

Hi. Thanks for this library.
The idea seems awesome. However, I still haven't got the chance to use it as I'm having issues with the conflicted dependencies stated above.

Also, I'm having issues installing pycocotools on windows10.

ERROR: Failed building wheel for pycocotools
Running setup.py clean for pycocotools
Failed to build pycocotools
Installing collected packages: pycocotools, opencv-python, fvcore, layoutparser
Attempting uninstall: pycocotools
Found existing installation: pycocotools 2.0
Uninstalling pycocotools-2.0:
Successfully uninstalled pycocotools-2.0
Running setup.py install for pycocotools ... error

It seems that for pycocotools the fix in windows 10 is:
cocodataset/cocoapi#169

Thanks for reporting this issue! There has been some updates for Detection, and the support for fvcore is somewhat problematic in the current version. I plan to remove the fvcore dependency in the next major release, which should be around within 1 month.

Any update on this?

Thank you!

[lolipopshock]

Thanks for reporting the issue for installing pycocotools on win10 - Let me try to fix that.

Speaking on the next major update, we are currently working on more features and it should be a huge update! Please stay tuned and we will release it very soon!