Optimize the pit collector script

[Chaoyingz]

Description

Optimize the pit collector script

Motivation and Context

I made a few changes to pit collector script:

• Split the download script into two parts: download data and normalize data
• Split _get_data_from_baostock method to improve readability
• Corresponding adjustment README.md and test_pit.py

How Has This Been Tested?

• Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.
• If you are adding a new feature, test on your own test scripts.

Screenshots of Test Results (if appropriate):

1. Pipeline test:
2. Your own tests:

PIT test results:

Types of changes

• Fix bugs
• Add new feature
• Update documentation

[SunsetWolf]

According to the PIT test results image you provided, I did the test and found some error messages and would like to get your help to fix it.

[Chaoyingz]

hi, @SunsetWolf, in qlib/data/data.py 1166 line:

try:
    return DatasetD.dataset(
        instruments, fields, start_time, end_time, freq, disk_cache, inst_processors=inst_processors
    )
except TypeError:
    return DatasetD.dataset(instruments, fields, start_time, end_time, freq, inst_processors=inst_processors)

It will catch the TypeError exception, but the exception is not captured in the screenshot you can provide, can you provide more information?

[SunsetWolf]

I would be more than happy to provide you with more detailed information about the error I received, and I hope this information is useful to you in helping me resolve this issue. Thanks!

[Chaoyingz]

Running this test requires initializing the calendar and instrument data.
Please execute the following command before running the test(you can find them from the comments in test_pit.py):

1. python scripts/get_data.py qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn

2. python scripts/data_collector/pit/collector.py download_data --source_dir ~/.qlib/stock_data/source/pit --start 2000-01-01 --end 2020-01-01 --interval quarterly --symbol_regex "^(600519|000725).*"

3. python scripts/data_collector/pit/collector.py normalize_data --interval quarterly --source_dir ~/.qlib/stock_data/source/pit --normalize_dir ~/.qlib/stock_data/source/pit_normalized

4. python scripts/dump_pit.py dump --csv_path ~/.qlib/stock_data/source/pit_normalized --qlib_dir ~/.qlib/qlib_data/cn_data --interval quarterly

[you-n-g]

Everything looks much simpler and cleaner than Qlib's initial version!
It looks great!
Thanks!