Problem: USD/CNY primitive steam doesn’t exist in TSN

[zolotokrylin]

Problem: USD/CNY primitive steam doesn’t exist in TSN

Originally posted by @zolotokrylin in https://github.com/truflation/truflation-stream/issues/23#issuecomment-1878030524

[j-dabrowski]

I'm currently working on this issue.

[j-dabrowski]

@joequant just trying to absorb all of this. I ran through the truflation-stream steps for setting up/using kwil-cli, and ran through the steps step 4 in truflation - is there an sql db I am to use here?

I read the discussion at #4 - I understand that my task is to modify the ingestors such that they can take USD/CNY data from a source (eg. yahoo finance), and kwil-cli can be used to add the ingested data to the kwil testnet. Am I on the right track here?

Thx for your help.

[joequant]

Yes that is write. There is a file in the examples which loads in data from investing.com and dumps it into a mysql database, and there is another example that loads in data from mysql and dumps it into kwill. What you need to do is write a new ingestor that loads in data from investing.com and dumps it into kwil.

@j-dabrowski

[zolotokrylin]

@j-dabrowski any questions or anything?

[j-dabrowski]

Hi @zolotokrylin I want to implement the example ingestors so I can test my own. Ie. get the pipeline details from an ingestor, use it to get, transform, and export the data object
Is this implemented already as I'm not seeing/understanding it

Will pick this up again after work today (6pm gmt+8)

Cheers

[zolotokrylin]

@joequant please help to address this question.

[joequant]

There are sample ingestors from - https://github.com/truflation/truflation/tree/dev/kwil/examples

The ingestor is the name of the program used to ingest data, and the pipeline details are coded in the python scripts

[joequant]

also I have created an existing stream callled om_investing_usd_cny_historical

kwil-cli database query 'select * from prices' -n kw1b2fd3bb87319429f656d45ad219f5

[j-dabrowski]

thanks @joequant
are you wanting me to push to truflation-stream or truflation?

[j-dabrowski]

Git auth issue... will try again tomorrow.

[zolotokrylin]

log: we had a call with @joequant and agreed to that Joe will attach the PR shortly

[j-dabrowski]

Not sure I have write access to truflation repo, getting 403 when pushing
Can someone take a look?