HAN's BLOG

The prediction of stock's price using Linear Regression for Machine Learning (2)

2021-12-04T00:00:00+00:00

Outline

This is purpose is prediction of stock’s price using ML with financial statements.

I have already gotten financial statements up to yahoo’s fianance web.This method have alreday been explained by this link. If you want how to get, you should check it. And this full idea has been already explained for kaggle.

The overall method will be carried out in the following …

Dats Preprocessing
Correlation for features
Modeling
Conclusion

Data Preprocessing

It is imfortant to analyze data for ML. I only use the recent value and numeric datas. Some example are B (bilion) -> 10^9 and etc… Link. And then, I load preprocessed data.

Previous Data Preprocessing
Additional Data Preprocessing

Additional Data Preprocessing

We have gotten price and inpormation of stocks, however, there are remained more to correlate somthings which are included to remove index and columns that are leak imformation. So, We need to correlate for porper value by reducing NaN values and adding or changing to numeric data.

Remove index

df = df[df['marketCap'].notna()]
df_index_null = pd.DataFrame(columns=['TotalNull', 'PercentOfNull'])
for ticker in df.index:
  temp_df = df.loc[ticker,:]
  count_null = temp_df.isnull().sum()
  percent_count_null = count_null/len(temp_df)
  df_index_null.loc[ticker, 'TotalNull'] = count_null
  df_index_null.loc[ticker, 'PercentOfNull'] = percent_count_null

remove_index = df_index_null[df_index_null['PercentOfNull']>0.5].index.tolist()

I hope to get the data over 50% about percent of null only. If you want to get value in wider range, you should just change 0.5 to 0.X.

unequal_in_index = [x for x in portfolio if x not in df.index.values.tolist()]
for l in unequal_in_index:
  portfolio.remove(l)
for tic in portfolio:
  if tic in remove_index:
    remove_index.remove(tic)
df = df.drop(remove_index, axis=0)
print('After removing index: ', len(df))

It should need to correlate same method for portfolio. Because portpolio as for valid train use individually full train wihch is index.

Remove columns

nulltotal = df.isnull().sum().sort_values(ascending=False)
nullpercent = ( df.isnull().sum() / len(df) ).sort_values(ascending=False)
nullpoint = pd.concat([nulltotal, nullpercent], axis=1, keys=['Total number of null',     'Percent of null'])
remove_cols = nullpercent[nullpercent >= 0.5].keys()
df = df.drop(remove_cols, axis=1)
newtotal = df.isnull().sum().sort_values(ascending=False)

In our analysis, we analyze the numeric datas for 84 financial statements. It is also needed to drop statements for leak information.

I hope to get the data over 50% about percent of null only. If you want to get value in wider range, you should just change 0.5 to 0.X.

Filling numeric data

numeric_missed = newtotal.index
for feature in numeric_missed:
  df[feature] = df[feature].fillna(0)

The missed data should just add 0 value.

Feature Engineering

from scipy.stats import norm, skew
numeric_feats = df.dtypes[df.dtypes != 'object'].index
skewed_feats = df[numeric_feats].apply(lambda x: skew(x)).sort_values(ascending=False)
high_skew = skewed_feats[abs(skewed_feats) > 0.5]

We could get features in high correlation.

The prediction of stock's price using Linear Regression for Machine Learning (1)

2021-11-28T00:00:00+00:00

Outline

1. Introduction

This is purpose is prediction of stock’s price using ML with financial statements.

The overall method will be carried out in the following …

Dats Preprocessing
Correlation for features
Modeling
Conclusion

Data Preprocessing

It is imfortant to analyze data for ML. I only use the recent value and numeric datas. Some example are B (bilion) -> 10^9 and etc… Link. And then, I load preprocessed data.

Previous Data Preprocessing
Additional Data Preprocessing

Previous Data Preprocessing

Load Financial statements of stock preprocessed

df_stats = pd.read_json(url+'/data_preprocessing/{0}_stats_element.json'.format(index_name))
df_addstats = pd.read_json(url+'/data_preprocessing/{0}_addstats_element.json'.format(index_name))
df_balsheets = pd.read_json(url+'/data_preprocessing/{0}_balsheets_element.json'.format(index_name))
df_income = pd.read_json(url+'/data_preprocessing/{0}_income_element.json'.format(index_name))
df_flow = pd.read_json(url+'/data_preprocessing/{0}_flow_element.json'.format(index_name))

Merge dataframe

df = pd.concat([df_stats, df_addstats, df_balsheets, df_income, df_flow], axis=1)

Check numeric datasets

from pandas.api.types import is_numeric_dtype
num_cols = [is_numeric_dtype(dtype) for dtype in df.dtypes]

Split data and test for correlation

from sklearn.model_selection import train_test_split
train_df_corr, test_df_corr = train_test_split(df, test_size=0.2)

Correlation for features and Heatmap

corrmat = train_df_corr.corr()
top_corr_features = corrmat.index[abs(corrmat['marketCap'])>0]
plt.figure(figsize=(13,10))
plt_corr = sns.heatmap(train_df_corr[top_corr_features].corr(), annot=True)

We can think how to consider features for changing the degree of correlation.

It is a value that does not take into account any degree.

There are many values that do not matter if viewed simply because there are tickers that contain insufficient information.Therefore, we need additional preprocessing to process in sufficient information.

Preprocessing with financial statements

2021-07-10T00:00:00+00:00

Preprocessing with financial statements

I have already gotten the financial statement of post.

It is enough from person to person. However I will analyze for meachine learning for regression.

I hope to get and change numeric data in statements.

If you want to understand what it is mean in financialk statements, you should know and search the defintion for google. It is not major of finance for me and already explained more easily than me for world.

I have used config file for root_url which is setted by myself and get index list using API of yahoo_fin.

And, I utilized class made by me. It is useful to preprocess and multiple strategy that it is updated for performance improvement curruntly.

yahoo_fin API

pip install yahoo_fin

You have to make directory(/data_preprocessing) before run below code.

Code to get preprocessing stock’s financial statements

import pandas as pd
import pandas_datareader as pdr
import yahoo_fin.stock_info as yfs

import json

def main(url='', index_list = ['AAPL'], index_name = 'dow', EACH = True, ALL = True, SHOW = True, SAVE = True):

    ## Configuration directory url
    url_data = url+'data_origin/'
    url_pre  = url+'data_preprocessing/'

    strategy = LongTermStrategy(url, filename) # Select Long term strategy

    ###################
    ## Preprocessing ##
    ###################

    if EACH == True:
        print('Preprocessing each Financial Statements')
        # Basic stats
        dict_basic = {}
        basic_fac = ['PER', 'PSR', 'PBR', 'PEG', 'forPER', 'Cap']
        print('\nEach factor preprocessing in Basic stats ('+ ' '.join(s for s in basic_fac), end=')\n')
        for fac in basic_fac:
            if fac == 'PER':
                df = strategy.get_PER()
            elif fac == 'PSR':
                df = strategy.get_PSR()
            elif fac == 'PBR':
                df = strategy.get_PBR()
            elif fac == 'PEG':
                df = strategy.get_PEG()
            elif fac == 'forPER':
                df = strategy.get_FORPER()
            elif fac == 'Cap':
                df = strategy.get_CAP()

            if SAVE == True:
                print('\n----------------------------- SAVE DATA  {} ---------------------------------\n'.format(fac))
                df.to_json(url_pre+'/{0}_{1}.json'.format(index_name, fac))
                df.to_csv(url_pre+'/{0}_{1}.csv'.format(index_name, fac))

            if SHOW == True:
                print(df)
            dict_basic[fac] = df

        # Additional stats
        dict_add = {}
        basic_fac = ['Beta', 'DivRate', 'ROE', 'ROA', 'PM', 'Cash', 'Debt']
        print('\nEach factor preprocessing in Additional stats ('+ ' '.join(s for s in basic_fac), end=')\n')
        for fac in basic_fac:
            if fac == 'Beta':
                df = strategy.get_Beta()
            elif fac == 'DivRate':
                df = strategy.get_DivRate()
            elif fac == 'ROE':
                df = strategy.get_ROE()
            elif fac == 'ROA':
                df = strategy.get_ROA()
            elif fac == 'PM':
                df = strategy.get_PM()
            elif fac == 'Cash':
                df = strategy.get_Cash()
            elif fac == 'Debt':
                df = strategy.get_Debt()

            if SAVE == True:
                print('\n----------------------------- SAVE DATA  {} ---------------------------------\n'.format(fac))
                df.to_json(url_pre+'/{0}_{1}.json'.format(index_name, fac))
                df.to_csv(url_pre+'/{0}_{1}.csv'.format(index_name, fac))

            if SHOW == True:
                print(df)
            dict_add[fac] = df

        # Balance sheets
        dict_bal = {}
        basic_fac = ['TA']
        print('\nEach factor preprocessing in Balance sheets ('+ ' '.join(s for s in basic_fac), end=')\n')
        for fac in basic_fac:
            if fac == 'TA':
                df = strategy.get_TA()

            if SAVE == True:
                print('\n----------------------------- SAVE DATA  {} ---------------------------------\n'.format(fac))
                df.to_json(url_pre+'/{0}_{1}.json'.format(index_name, fac))
                df.to_csv(url_pre+'/{0}_{1}.csv'.format(index_name, fac))

            if SHOW == True:
                print(df)
            dict_bal[fac] = df

        # Income sheets
        dict_income = {}
        basic_fac = ['TR']
        print('\nEach factor preprocessing in Income sheets ('+ ' '.join(s for s in basic_fac), end=')\n')
        for fac in basic_fac:
            if fac == 'TR':
                df = strategy.get_TR()

            if SAVE == True:
                print('\n----------------------------- SAVE DATA  {} ---------------------------------\n'.format(fac))
                df.to_json(url_pre+'/{0}_{1}.json'.format(index_name, fac))
                df.to_csv(url_pre+'/{0}_{1}.csv'.format(index_name, fac))

            if SHOW == True:
                print(df)
            dict_income[fac] = df

        # Cash flow
        dict_flow = {}
        basic_fac = ['DIV', 'ISS']
        print('\nEach factor preprocessing in Cash flow ('+ ' '.join(s for s in basic_fac), end=')\n')
        for fac in basic_fac:
            if fac == 'DIV':
                df = strategy.get_DIV()
            elif fac == 'ISS':
                df = strategy.get_ISS()

            if SAVE == True:
                print('\n----------------------------- SAVE DATA  {} ---------------------------------\n'.format(fac))
                df.to_json(url_pre+'/{0}_{1}.json'.format(index_name, fac))
                df.to_csv(url_pre+'/{0}_{1}.csv'.format(index_name, fac))

            if SHOW == True:
                print(df)
            dict_flow[fac] = df

    if ALL == True:
        stats_name = ['stats', 'addstats', 'balsheets', 'income', 'flow']
        print('\nPreprocessing element in ('+ ' '.join(s for s in stats_name), end=')\n')
        for stat in stats_name:
            if stat == 'stats':
                df = strategy.get_stats(preprocessing = True)
            elif stat == 'addstats':
                df = strategy.get_addstats(preprocessing = True)
            elif stat == 'balsheets':
                df = strategy.get_balsheets_element(index_list)
            elif stat == 'income':
                df = strategy.get_income_element(index_list)
            elif stat == 'flow':
                df = strategy.get_flow_element(index_list)

            if SAVE == True:
                print('\n----------------------------- SAVE DATA  {} ---------------------------------\n'.format(stat))
                print('')
                df.to_json(url_pre+'/{0}_{1}_element.json'.format(index_name, stat))
                df.to_csv(url_pre+'/{0}_{1}_element.csv'.format(index_name, stat))

            if SHOW == True:
                print(df)

if __name__=='__main__':
    from class_Strategy import LongTermStrategy
    with open('config/config.json', 'r') as f:
        config = json.load(f)
    root_url = config['root_dir']
    dow_list = yfs.tickers_dow()
    filename = ''
    s= input("Choice of stock's list (dow, sp500, nasdaq, other, all, selected): ")
    if s == 'dow':
        dow_list = yfs.tickers_dow()
        filename = 'dow'
    elif s == 'sp500':
        filename = 'sp500'
        dow_list = yfs.tickers_sp500()
    elif s == 'nasdaq':
        filename = 'nasdaq'
        dow_list = yfs.tickers_nasdaq()
    elif s == 'other':
        filename = 'other'
        dow_list = yfs.tickers_other()
    elif s == 'all':
        filename = 'all'
        dow_list_1 = yfs.tickers_nasdaq()
        dow_list_2 = yfs.tickers_other()
        dow_list = dow_list_1 + dow_list_2
    elif s == 'selected':
        filename = 'selected'
        url = 'data_ForTrading/selected_ticker.json'
        temp_pd = pd.read_json(url)
        temp_pd = temp_pd['Ticker']
        dow_list = temp_pd.values.tolist()

    main(url=root_url, index_list = dow_list, index_name = filename, EACH=True, ALL=True, SAVE=True, SHOW=False)

else:
    pass

It is needed what to set various factor like PER PBR and so on.

And, we can choose parameter what you want in main funtion.

Result

It is example for dow index not to show dataframes which are setted as parameter is False.

Choice of stock's list (dow, sp500, nasdaq, other, all, selected): dow
Preprocessing each Financial Statements
   
Each factor preprocessing in Basic stats (PER PSR PBR PEG forPER Cap)

----------------------------- SAVE DATA  PER ---------------------------------

----------------------------- SAVE DATA  PSR ---------------------------------

----------------------------- SAVE DATA  PBR ---------------------------------

   ----------------------------- SAVE DATA  PEG ---------------------------------

----------------------------- SAVE DATA  forPER ---------------------------------

----------------------------- SAVE DATA  Cap ---------------------------------

Each factor preprocessing in Additional stats (Beta DivRate ROE ROA PM Cash Debt)

----------------------------- SAVE DATA  Beta ---------------------------------

----------------------------- SAVE DATA  DivRate ---------------------------------

----------------------------- SAVE DATA  ROE ---------------------------------

----------------------------- SAVE DATA  ROA ---------------------------------

----------------------------- SAVE DATA  PM ---------------------------------

----------------------------- SAVE DATA  Cash ---------------------------------

----------------------------- SAVE DATA  Debt ---------------------------------

Each factor preprocessing in Balance sheets (TA)

----------------------------- SAVE DATA  TA ---------------------------------

Each factor preprocessing in Income sheets (TR)

----------------------------- SAVE DATA  TR ---------------------------------

Each factor preprocessing in Cash flow (DIV ISS)

----------------------------- SAVE DATA  DIV ---------------------------------

----------------------------- SAVE DATA  ISS ---------------------------------

Preprocessing element in (stats addstats balsheets income flow)

----------------------------- SAVE DATA  stats ---------------------------------

----------------------------- SAVE DATA  addstats ---------------------------------

For balance sheets
100%|████████████████████████████████████████████████████████████████████| 30/30 [00:01<00:00, 28.21it/s]

----------------------------- SAVE DATA  balsheets ---------------------------------

For income statements
100%|████████████████████████████████████████████████████████████████████| 30/30 [00:00<00:00, 45.14it/s]

----------------------------- SAVE DATA  income ---------------------------------

For cash flow
100%|████████████████████████████████████████████████████████████████████| 30/30 [00:00<00:00, 47.32it/s]

----------------------------- SAVE DATA  flow ---------------------------------

Conclusion

Let us know how to preprocess stock’s financi statements in index.

If you want to use this code, I’m very sorry that you should change code and make directory for data a little bit.

I should appreciate and refer for many blog on google. Thanks a lot.

If you satisfied this post you should check Github and please Star :)

How to get stock's financial statements?

2021-07-04T00:00:00+00:00

Stock’s financial statements

It is important for stock to get the price as well as financial statements

This is presented typically PER and ROA etc. The other’s factor like income statement and cash flow based on yahoo finance site using yahoo_fin API with python and refer to yahoo_fin documation.

yahoo_fin API

pip install yahoo_fin

I use this both API in condition.

Code to get stock’s financial statements

import yahoo_fin.stock_info as yfs
import pandas as pd

from tqdm import tqdm
import json

def main(url, index_list, index_name):

    ## Configuration directory url
    url_data = url+'data_origin/'

    # For check the table of one ticker like aapl
    aapl_quote = yfs.get_quote_table('aapl')
    aapl_val = yfs.get_stats_valuation('aapl')
    aapl_ext = yfs.get_stats('aapl')
    aapl_sheet = yfs.get_balance_sheet('aapl')
    aapl_income = yfs.get_income_statement('aapl')
    aapl_flow = yfs.get_cash_flow('aapl')

    print('*'*100)
    print('quote')
    print(aapl_quote)
    print('*'*100)
    print('basic')
    print(aapl_val)
    print('*'*100)
    print('additional')
    print(aapl_ext)
    print('*'*100)
    print('balance sheets')
    print(aapl_sheet)
    print(len(aapl_sheet.columns))
    print('*'*100)
    print('income statements')
    print(aapl_income)
    print('*'*100)
    print('cash flow')
    print(aapl_flow)
    print('*'*100)

    # Get data in the current  olumn for each stock's valuation table
    dow_stats = {}
    dow_addstats = {}
    dow_balsheets = {}
    dow_income = {}
    dow_flow = {}

    error_symbols = []
    for ticker in tqdm(index_list):
        try:
            # Getteing Summary
            basic = yfs.get_stats_valuation(ticker)
            basic =basic.iloc[:,:2]
            basic.columns = ['Attribute', 'Recent']
            dow_stats[ticker] = basic

            # Getting additioanl stats
            add = yfs.get_stats(ticker)
            add.columns = ['Attribute', 'Value']
            dow_addstats[ticker] = add

            # Getting balance sheets
            sheets = yfs.get_balance_sheet(ticker)
            dow_balsheets[ticker] = sheets

            # Getting income statements
            income = yfs.get_income_statement(ticker)
            dow_income[ticker] = income

            # Getting cash flow statements
            flow = yfs.get_cash_flow(ticker)
            dow_flow[ticker] = flow

        except:
            error_symbols.append(ticker)
            print('Error ticker: ', ticker)

    print('error symol: ', error_symbols)

    for ticker in dow_balsheets.keys():
        leng = len(dow_balsheets[ticker].columns)
        dow_balsheets[ticker].columns = ['Before_'+str(i) for i in range(0,leng)]
        dow_balsheets[ticker] = dow_balsheets[ticker].rename(columns={'Before_0': 'Recent'})

    for ticker in dow_income.keys():
        leng = len(dow_income[ticker].columns)
        dow_income[ticker].columns = ['Before_'+str(i) for i in range(0,leng)]
        dow_income[ticker] = dow_income[ticker].rename(columns={'Before_0': 'Recent'})

    for ticker in dow_flow.keys():
        leng = len(dow_flow[ticker].columns)
        dow_flow[ticker].columns = ['Before_'+str(i) for i in range(0,leng)]
        dow_flow[ticker] = dow_flow[ticker].rename(columns={'Before_0': 'Recent'})


    combined_stats = pd.concat(dow_stats)
    combined_stats = combined_stats.reset_index()
    combined_stats = combined_stats.rename(columns={'level_0': 'Ticker'})

    combined_addstats = pd.concat(dow_addstats)
    combined_addstats = combined_addstats.reset_index()
    combined_addstats = combined_addstats.rename(columns={'level_0': 'Ticker'})

    combined_balsheets = pd.concat(dow_balsheets)
    combined_balsheets = combined_balsheets.reset_index()
    combined_balsheets = combined_balsheets.rename(columns={'level_0': 'Ticker'})

    combined_income = pd.concat(dow_income)
    combined_income = combined_income.reset_index()
    combined_income = combined_income.rename(columns={'level_0': 'Ticker'})

    combined_flow = pd.concat(dow_flow)
    combined_flow = combined_flow.reset_index()
    combined_flow = combined_flow.rename(columns={'level_0': 'Ticker'})

    del combined_stats['level_1']
    del combined_addstats['level_1']

    print(combined_stats)
    print(combined_addstats)
    print(combined_balsheets)
    print(combined_income)
    print(combined_flow)

    list_stats = ['stats', 'addstats', 'balsheets', 'income', 'flow']
    for s in list_stats:
        url_FS = url_data+'FS_{0}_{1}'.format(index_name, s)
        if s == 'stats':
            df = combined_stats
        elif s == 'addstats':
            df = combined_addstats
        elif s == 'balsheets':
            df = combined_balsheets
        elif s == 'income':
            df = combined_income
        elif s == 'flow':
            df = combined_flow
        df.to_json(url_FS+'.json')
        df.to_csv(url_FS+'.csv')

if __name__ == '__main__':
    with open('../config/config.json', 'r') as f:
        config = json.load(f)
    root_url = config['root_dir']

    filename = input("Choice of stock's list (dow, sp500, nasdaq, other, selected): ")

    # Get list of Dow tickers
    dow_list = yfs.tickers_dow()
    if filename == 'dow':
        dow_list = yfs.tickers_dow()
    elif filename == 'sp500':
        dow_list = yfs.tickers_sp500()
    elif filename == 'nasdaq':
        dow_list = yfs.tickers_nasdaq()
    elif filename == 'other':
        dow_list = yfs.tickers_other()
    elif filename == 'selected':
        url = root_url+'/data_ForTrading/selected_ticker.json'
        temp_pd = pd.read_json(url)
        temp_pd = temp_pd['Ticker']
        dow_list = temp_pd.values.tolist()

    print(dow_list)

    main(url=root_url, index_list = dow_list, index_name = filename)

else:
    pass

FIrst of all, we can check how to get statement by yahoo_fin API for raw.

And then, I select the index of stock for us using API.

The dataframe is made by continus ticker with adjsuting change refer to ducumation of yahoo_fin API.

Result

one example result by ticker of AAPL for raw yahoo_fin API

****************************************************************************************************
basic
                             0      1
0      Market Cap (intraday) 5  2.34T
1           Enterprise Value 3   2.3T
2                 Trailing P/E  31.46
3                Forward P/E 1  26.16
4  PEG Ratio (5 yr expected) 1   1.45
5            Price/Sales (ttm)   7.18
6             Price/Book (mrq)  33.76
7   Enterprise Value/Revenue 3   7.07
8    Enterprise Value/EBITDA 7  23.05
****************************************************************************************************
additional
                                         Attribute         Value
0                                Beta (5Y Monthly)          1.21
1                                 52-Week Change 3           NaN
2                          S&P500 52-Week Change 3           NaN
3                                   52 Week High 3        145.09
4                                    52 Week Low 3         89.14
5                          50-Day Moving Average 3         128.8
6                         200-Day Moving Average 3        129.16
7                              Avg Vol (3 month) 3        82.31M
8                               Avg Vol (10 day) 3        64.61M
9                             Shares Outstanding 5        16.69B
10                    Implied Shares Outstanding 6           NaN
11                                           Float        16.67B
12                            % Held by Insiders 1         0.07%
13                        % Held by Institutions 1        58.69%
14                   Shares Short (May 27, 2021) 4       123.12M
15                    Short Ratio (May 27, 2021) 4          1.36
16               Short % of Float (May 27, 2021) 4         0.74%
17  Short % of Shares Outstanding (May 27, 2021) 4         0.74%
18       Shares Short (prior month Apr 29, 2021) 4        82.71M
19                  Forward Annual Dividend Rate 4          0.88
20                 Forward Annual Dividend Yield 4         0.66%
21                 Trailing Annual Dividend Rate 3          0.82
22                Trailing Annual Dividend Yield 3         0.60%
23                 5 Year Average Dividend Yield 4          1.34
24                                  Payout Ratio 4        18.34%
25                                 Dividend Date 3  May 12, 2021
26                              Ex-Dividend Date 4  May 06, 2021
27                             Last Split Factor 2           4:1
28                               Last Split Date 3  Aug 30, 2020
29                                Fiscal Year Ends  Sep 25, 2020
30                       Most Recent Quarter (mrq)  Mar 26, 2021
31                                   Profit Margin        23.45%
32                          Operating Margin (ttm)        27.32%
33                          Return on Assets (ttm)        16.90%
34                          Return on Equity (ttm)       103.40%
35                                   Revenue (ttm)       325.41B
36                         Revenue Per Share (ttm)         19.14
37                  Quarterly Revenue Growth (yoy)        53.60%
38                              Gross Profit (ttm)       104.96B
39                                          EBITDA        99.82B
40                  Net Income Avi to Common (ttm)        76.31B
41                               Diluted EPS (ttm)          4.45
42                 Quarterly Earnings Growth (yoy)       110.10%
43                                Total Cash (mrq)        69.83B
44                      Total Cash Per Share (mrq)          4.18
45                                Total Debt (mrq)       134.74B
46                         Total Debt/Equity (mrq)        194.78
47                             Current Ratio (mrq)          1.14
48                      Book Value Per Share (mrq)          4.15
49                       Operating Cash Flow (ttm)        99.59B
50                    Levered Free Cash Flow (ttm)        80.12B
****************************************************************************************************
balance sheets
endDate                    2020-09-26    2019-09-28    2018-09-29    2017-09-30
Breakdown
totalLiab                258549000000  248028000000  258578000000  241272000000
totalStockholderEquity    65339000000   90488000000  107147000000  134047000000
otherCurrentLiab          47867000000   43242000000   39293000000   38099000000
totalAssets              323888000000  338516000000  365725000000  375319000000
commonStock               50779000000   45174000000   40201000000   35867000000
otherCurrentAssets        11264000000   12352000000   12087000000   13936000000
retainedEarnings          14966000000   45898000000   70400000000   98330000000
otherLiab                 46108000000   50503000000   48914000000   43251000000
treasuryStock              -406000000    -584000000   -3454000000    -150000000
otherAssets               33952000000   32978000000   22283000000   18177000000
cash                      38016000000   48844000000   25913000000   20289000000
totalCurrentLiabilities  105392000000  105718000000  115929000000  100814000000
shortLongTermDebt          8773000000   10260000000    8784000000    6496000000
otherStockholderEquity     -406000000    -584000000   -3454000000    -150000000
propertyPlantEquipment    45336000000   37378000000   41304000000   33783000000
totalCurrentAssets       143713000000  162819000000  131339000000  128645000000
longTermInvestments      100887000000  105341000000  170799000000  194714000000
netTangibleAssets         65339000000   90488000000  107147000000  134047000000
shortTermInvestments      52927000000   51713000000   40388000000   53892000000
netReceivables            37445000000   45804000000   48995000000   35673000000
longTermDebt              98667000000   91807000000   93735000000   97207000000
inventory                  4061000000    4106000000    3956000000    4855000000
accountsPayable           42296000000   46236000000   55888000000   44242000000
****************************************************************************************************
income statements
endDate                              2020-09-26    2019-09-28    2018-09-29    2017-09-30
Breakdown
researchDevelopment                 18752000000   16217000000   14236000000   11581000000
effectOfAccountingCharges                  None          None          None          None
incomeBeforeTax                     67091000000   65737000000   72903000000   64089000000
minorityInterest                           None          None          None          None
netIncome                           57411000000   55256000000   59531000000   48351000000
sellingGeneralAdministrative        19916000000   18245000000   16705000000   15261000000
grossProfit                        104956000000   98392000000  101839000000   88186000000
ebit                                66288000000   63930000000   70898000000   61344000000
operatingIncome                     66288000000   63930000000   70898000000   61344000000
otherOperatingExpenses                     None          None          None          None
interestExpense                     -2873000000   -3576000000   -3240000000   -2323000000
extraordinaryItems                         None          None          None          None
nonRecurring                               None          None          None          None
otherItems                                 None          None          None          None
incomeTaxExpense                     9680000000   10481000000   13372000000   15738000000
totalRevenue                       274515000000  260174000000  265595000000  229234000000
totalOperatingExpenses             208227000000  196244000000  194697000000  167890000000
costOfRevenue                      169559000000  161782000000  163756000000  141048000000
totalOtherIncomeExpenseNet            803000000    1807000000    2005000000    2745000000
discontinuedOperations                     None          None          None          None
netIncomeFromContinuingOps          57411000000   55256000000   59531000000   48351000000
netIncomeApplicableToCommonShares   57411000000   55256000000   59531000000   48351000000
****************************************************************************************************
cash flow
endDate                                 2020-09-26   2019-09-28   2018-09-29   2017-09-30
Breakdown
investments                             5335000000  58093000000  30845000000 -33542000000
changeToLiabilities                    -1981000000  -2548000000   9172000000   8373000000
totalCashflowsFromInvestingActivities  -4289000000  45896000000  16066000000 -46446000000
netBorrowings                           2499000000  -7819000000    432000000  29014000000
totalCashFromFinancingActivities      -86820000000 -90976000000 -87876000000 -17974000000
changeToOperatingActivities              881000000   -896000000  30016000000  -8480000000
issuanceOfStock                          880000000    781000000    669000000    555000000
netIncome                              57411000000  55256000000  59531000000  48351000000
changeInCash                          -10435000000  24311000000   5624000000   -195000000
repurchaseOfStock                     -75992000000 -69714000000 -75265000000 -34774000000
totalCashFromOperatingActivities       80674000000  69391000000  77434000000  64225000000
depreciation                           11056000000  12547000000  10903000000  10157000000
otherCashflowsFromInvestingActivities   -791000000  -1078000000   -745000000   -124000000
dividendsPaid                         -14081000000 -14119000000 -13712000000 -12769000000
changeToInventory                       -127000000   -289000000    828000000  -2723000000
changeToAccountReceivables              6917000000    245000000  -5322000000  -2093000000
otherCashflowsFromFinancingActivities   -126000000   -105000000   -105000000   -105000000
changeToNetincome                       6517000000   5076000000 -27694000000  10640000000
capitalExpenditures                    -7309000000 -10495000000 -13313000000 -12451000000
****************************************************************************************************

Then, I change adjusting data frame for analysis.

It is one example for dow index

   Ticker                    Attribute Recent
   AAPL      Market Cap (intraday) 5  2.34T
   AAPL           Enterprise Value 3   2.3T
   AAPL                 Trailing P/E  31.46
   AAPL                Forward P/E 1  26.16
   AAPL  PEG Ratio (5 yr expected) 1   1.45
..     ...                          ...    ...
  WMT  PEG Ratio (5 yr expected) 1   3.26
  WMT            Price/Sales (ttm)   0.70
  WMT             Price/Book (mrq)   5.02
  WMT   Enterprise Value/Revenue 3   0.76
  WMT    Enterprise Value/EBITDA 7  10.78

[270 rows x 3 columns]
    Ticker                     Attribute   Value
    AAPL             Beta (5Y Monthly)    1.21
    AAPL              52-Week Change 3     NaN
    AAPL       S&P500 52-Week Change 3     NaN
    AAPL                52 Week High 3  145.09
    AAPL                 52 Week Low 3   89.14
...     ...                           ...     ...
  WMT       Total Debt/Equity (mrq)   74.74
  WMT           Current Ratio (mrq)    0.95
  WMT    Book Value Per Share (mrq)   27.93
  WMT     Operating Cash Flow (ttm)  31.91B
  WMT  Levered Free Cash Flow (ttm)  17.24B

[1530 rows x 3 columns]
   Ticker               Breakdown        Recent      Before_1      Before_2      Before_3
   AAPL               totalLiab  2.585490e+11  2.480280e+11  2.585780e+11  2.412720e+11
   AAPL  totalStockholderEquity  6.533900e+10  9.048800e+10  1.071470e+11  1.340470e+11
   AAPL        otherCurrentLiab  4.786700e+10  4.324200e+10  3.929300e+10  3.809900e+10
   AAPL             totalAssets  3.238880e+11  3.385160e+11  3.657250e+11  3.753190e+11
   AAPL             commonStock  5.077900e+10  4.517400e+10  4.020100e+10  3.586700e+10
..     ...                     ...           ...           ...           ...           ...
  WMT       netTangibleAssets  4.704200e+10  3.839600e+10  3.551500e+10  5.962700e+10
  WMT          netReceivables  6.516000e+09  6.284000e+09  6.283000e+09  5.614000e+09
  WMT            longTermDebt  4.165000e+10  4.441000e+10  4.394800e+10  3.023100e+10
  WMT               inventory  4.494900e+10  4.443500e+10  4.426900e+10  4.378300e+10
  WMT         accountsPayable  4.914100e+10  4.697300e+10  4.706000e+10  4.609200e+10

[789 rows x 6 columns]
   Ticker                          Breakdown        Recent      Before_1      Before_2      Before_3
   AAPL                researchDevelopment   18752000000   16217000000   14236000000   11581000000
   AAPL          effectOfAccountingCharges          None          None          None          None
   AAPL                    incomeBeforeTax   67091000000   65737000000   72903000000   64089000000
   AAPL                   minorityInterest          None          None          None          None
   AAPL                          netIncome   57411000000   55256000000   59531000000   48351000000
..     ...                                ...           ...           ...           ...           ...
  WMT                      costOfRevenue  420315000000  394605000000  385301000000  373396000000
  WMT         totalOtherIncomeExpenseNet   -6384000000   -1352000000  -10497000000   -5814000000
  WMT             discontinuedOperations          None          None          None          None
  WMT         netIncomeFromContinuingOps   13706000000   15201000000    7179000000   10523000000
  WMT  netIncomeApplicableToCommonShares   13510000000   14881000000    6670000000    9862000000

[660 rows x 6 columns]
   Ticker                              Breakdown        Recent      Before_1      Before_2      Before_3
   AAPL                            investments  5.335000e+09  5.809300e+10  3.084500e+10 -3.354200e+10
   AAPL                    changeToLiabilities -1.981000e+09 -2.548000e+09  9.172000e+09  8.373000e+09
   AAPL  totalCashflowsFromInvestingActivities -4.289000e+09  4.589600e+10  1.606600e+10 -4.644600e+10
   AAPL                          netBorrowings  2.499000e+09 -7.819000e+09  4.320000e+08  2.901400e+10
   AAPL       totalCashFromFinancingActivities -8.682000e+10 -9.097600e+10 -8.787600e+10 -1.797400e+10
..     ...                                    ...           ...           ...           ...           ...
  WMT                      changeToInventory -2.395000e+09 -3.000000e+08 -1.311000e+09 -1.400000e+08
  WMT             changeToAccountReceivables -1.086000e+09  1.540000e+08 -3.680000e+08 -1.074000e+09
  WMT  otherCashflowsFromFinancingActivities -1.670000e+09 -1.463000e+09 -1.060000e+09 -4.018000e+09
  WMT                      changeToNetincome  3.440000e+09 -2.860000e+08  1.011000e+10  4.703000e+09
  WMT                    capitalExpenditures -1.026400e+10 -1.070500e+10 -1.034400e+10 -1.005100e+10

[544 rows x 6 columns]

Conclusion

Let us know how to get stock’s financi statements in index.

If you want to use this code, I’m very sorry that you should change code and make directory for data a little bit.

I should appreciate and refer for many blog on google. Thanks a lot.

If you satisfied this post you should check Github and please Star :)

How to get stock's price?

2021-06-28T00:00:00+00:00

Stock’s price

This post introduce for us to get stock price using API with python.

pandas_datareader API

it is very useful API for free to be able to get at any web site.

pip install pandas-datareader

yahoo_fin API

It is very useful API for free to be able to get only yahoo finance in additional to get financial statements.

pip install yahoo_fin

I use this both API in condition.

This APIs have advantage for free but, it is not official for website like yahoo finance.

Therefore, they were sometimes blocked. For free user like us, we use according to the situation and participate updating library :)

Code to get stock’s price

import pandas as pd
import pandas_datareader as pdr
import yahoo_fin.stock_info as yfs

from tqdm import tqdm
import datetime
import json

def main(url, index_list, index_name, start, end, run_yfs=False, run_pdr=True):

    ## Configuration directory url
    url_data = url+'data_origin/'

    print('Example Apple inc')
    if run_pdr:
        df_aapl = pdr.DataReader('AAPL', 'yahoo', start_day, today)

    if run_yfs:
        start_day.strftime('%m/%d/%y')
        today.strftime('%m/%d/%y')
        df_aapl = yfs.get_data('AAPL',start_date=start_day, end_date = today)
        df_aapl = df_aapl.reset_index()
        df_aapl = df_aapl.rename(columns={'open':'Open', 'index':'Date', 'high':'High','low':'Low','close':'Close','adjclose':'Adj Close','volume':'Volume','ticker':'Ticker'})
        df_aapl = df_aapl[['Ticker','Date','High','Low','Open','Close','Volume','Adj Close']]
    print(df_aapl)

    df_recent = pd.DataFrame({'Recent_price': []})
    etf_values = {}
    error_symbols = []
    for ticker in tqdm(index_list):
        try:
            if run_pdr:
                df = pdr.DataReader(ticker,'yahoo', start_day, today)
            if run_yfs:
                df = yfs.get_data(ticker,start_date=start_day, end_date = today)
                df = df.reset_index()
                df = df.rename(columns={'open':'Open', 'index':'Date', 'high':'High','low':'Low','close':'Close','adjclose':'Adj Close','volume':'Volume','ticker':'Ticker'})
                df = df[['Ticker','Date','High','Low','Open','Close','Volume','Adj Close']]
            df_recent.loc[ticker, 'Recent_price'] = yfs.get_live_price(ticker)
            etf_values[ticker] = df
        except:
            error_symbols.append(ticker)
            print('Error: ', error_symbols)
            df_recent.loc[ticker, 'Recent_price'] = yfs.get_live_price(ticker)

    if run_pdr:
        combined_value = pd.concat(etf_values)
        combined_value = combined_value.reset_index()
        combined_value= combined_value.rename(columns={'level_0': 'Ticker'})
    if run_yfs:
        combined_value = pd.concat(etf_values)
        combined_value = combined_value.reset_index()
        combined_value = combined_value.drop(['level_0','level_1'], axis=1)

    print(combined_value)
    print(df_recent)

    url_value_data = url_data+'FS_{0}_Value'.format(index_name)
    combined_value.to_json(url_value_data+'.json')
    combined_value.to_csv(url_value_data+'.csv')

    url_recent_value_data = url_data+'FS_{0}_Recent_Value'.format(index_name)
    df_recent.to_json(url_recent_value_data+'.json')
    df_recent.to_csv(url_recent_value_data+'.csv')

if __name__ == '__main__':
    with open('../config/config.json', 'r') as f:
        config = json.load(f)
    root_url = config['root_dir']

    filename= input("Choice of stock's list (dow, sp500, nasdaq, other, selected): ")

    dow_list = yfs.tickers_dow()
    if filename == 'dow':
        dow_list = yfs.tickers_dow()
    elif filename == 'sp500':
        dow_list = yfs.tickers_sp500()
    elif filename == 'nasdaq':
        dow_list = yfs.tickers_nasdaq()
    elif filename == 'other':
        dow_list = yfs.tickers_other()
    elif filename == 'selected':
        url = root_url+'/data_ForTrading/selected_ticker.json'
        temp_pd = pd.read_json(url)
        temp_pd = temp_pd['Ticker']
        dow_list = temp_pd.values.tolist()

    print(dow_list)

    ## Define time of start to end ##
    #td_1y = datetime.timedelta(weeks=52/2)
    start_day = datetime.datetime(2010,1,1)
    today = datetime.datetime.now()
    #start_day = today-td_1y

    print('Price of stcok in {0} for date series and recent price'.format(filename))
    main(url=root_url, index_list = dow_list, index_name=filename, start= start_day, end = today, run_yfs = True, run_pdr=False)

else:
    pass

We can get in condition of setting from start day to end day (today) and Recent price of live.

And, we’re also able to get index of stock like Dow and S&P500 etc.

Result

The example result of upper code at dow index.

     Ticker       Date        High         Low        Open       Close       Volume   Adj Close
0       AAPL 2009-12-31    7.619643    7.520000    7.611786    7.526072  352410800.0    6.471693
1       AAPL 2010-01-04    7.660714    7.585000    7.622500    7.643214  493729600.0    6.572423
2       AAPL 2010-01-05    7.699643    7.616071    7.664286    7.656428  601904800.0    6.583786
3       AAPL 2010-01-06    7.686786    7.526786    7.656428    7.534643  552160000.0    6.479064
4       AAPL 2010-01-07    7.571429    7.466072    7.562500    7.520714  477131200.0    6.467088
...      ...        ...         ...         ...         ...         ...          ...         ...
84407    WMT 2021-06-21  136.750000  135.350006  135.660004  136.399994    6925000.0  136.399994
84408    WMT 2021-06-22  137.720001  136.229996  136.229996  137.029999    6369000.0  137.029999
84409    WMT 2021-06-23  136.990005  135.929993  136.500000  135.960007    6463500.0  135.960007
84410    WMT 2021-06-24  137.240005  136.100006  136.169998  136.910004    7814700.0  136.910004
84411    WMT 2021-06-25  138.860001  136.919998  137.130005  138.529999    9544400.0  138.529999

[84412 rows x 8 columns]
     Recent_price
AAPL    133.110001
AMGN    242.679993
AXP     169.449997
BA      248.380005
CAT     216.309998
CRM     241.869995
CSCO     53.060001
CVX     107.300003
DIS     178.350006
DOW      63.290001
GS      368.769989
HD      313.630005
HON     218.740005
IBM     146.839996
INTC     55.910000
JNJ     164.210007
JPM     154.050003
KO       54.320000
MCD     232.419998
MMM     194.750000
MRK      77.199997
MSFT    265.019989
NKE     154.350006
PG      134.919998
TRV     151.720001
UNH     404.950012
V       237.320007
VZ       56.380001
WBA      52.160000
WMT     138.529999

Conclusion

Let us know how to get stock’s price in index.

If you want to use this code, I’m very sorry that you should change code and make directory for data a little bit.

I should appreciate and refer for many blog on google. Thanks a lot.

If you satisfied this post you should check Github and please Star :)

Download financial news of yahoo with python

2021-06-27T00:00:00+00:00

Financial news

It is so important statemennts for study stock.

I’m simply check only headline and summary.

It is expected to study natural language processing (NLP) to future :)

Class code

# class_Strategy.py
# import yahoo_fin import news as ynews

class NLPStrategy:
    def __init__(self, url, etfname, Offline = False):
        self.url = url
        self.etfname = etfname
        self.Offline = Offline

    def get_news_title(self, ticker = 'AAPL'):
        if self.Offline == True:
            url_news = self.url+'/data_origin/FS_'+self.etfname+'_title.json'
            with open (url_news, 'r') as f:
                title = json.load(f)
        else:
            title = []
            news = ynews.get_yf_rss(ticker)
            for k,v in enumerate(news):
                title.append(v['title'])

        return title

    def get_news_summary(self, ticker = 'AAPL'):
        if self.Offline == True:
            url_news = self.url+'/data_origin/FS_'+self.etfname+'_summary.json'
            with open (url_news, 'r') as f:
                title = json.load(f)

        else:
            summary = []
            news = ynews.get_yf_rss(ticker)
            for k,v in enumerate(news):
                summary.append(v['summary'])

            return summary

yahoo_fin API is very useful for a lot of sotck’s information. If you are interesting this api please check and use page

I choose data list divided online and offline.

Online test can download only one ticker and then gather using loop in main code.
Offline test can download index stock after pre-download below code for choice.

Choice Ticker

import json
import pandas as pd
import yahoo_fin.stock_info as yfs
from yahoo_fin import news as ynews

from tqdm import tqdm

def main(stock_list):

    etf_list = yfs.tickers_dow()
    if stock_list == 'dow':
        etf_list = yfs.tickers_dow()
        filename = 'dow'
    elif stock_list == 'sp500':
        filename = 'sp500'
        etf_list = yfs.tickers_sp500()
    elif stock_list == 'nasdaq':
        filename = 'nasdaq'
        etf_list = yfs.tickers_nasdaq()
    elif stock_list == 'other':
        filename = 'other'
        etf_list = yfs.tickers_other()
    elif stock_list == 'selected':
        filename = 'selected'
        url = '/Users/hanseopark/Work/stock/data_ForTrading/selected_ticker.json'
        temp_pd = pd.read_json(url)
        temp_pd = temp_pd['Ticker']
        etf_list = temp_pd.values.tolist()

    print(etf_list)

    error_symbols = []

    total_news_title = {}
    total_news_summary = {}

    for ticker in tqdm(etf_list):
        list_title = []
        list_summary = []
        try:
            list_news= ynews.get_yf_rss(ticker)
            for k,v in enumerate(list_news):
                list_title.append(v['title'])
                list_summary.append(v['summary'])

        except:
            error_symbols.append(ticker)
        total_news_title[ticker] = list_title
        total_news_summary[ticker] = list_title

    #print(total_news_title)
    #print(total_news_summary)
    print(error_symbols)

    url_title = '/Users/hanseopark/Work/stock/data_origin/FS_{0}_title.json'.format(filename)
    url_summary = '/Users/hanseopark/Work/stock/data_origin/FS_{0}_summary.json'.format(filename)
    with open (url_title, 'w') as f:
        json.dump(total_news_title, f)

    with open (url_summary, 'w') as f:
        json.dump(total_news_summary, f)

    ## temp reading
#    with open (url, 'r') as f:
#        temp = json.load(f)
#    print(temp)

if __name__ == '__main__':
    s= input("Choice of stock's list (dow, sp500, nasdaq, other, selected): ")

    print("News' title and summary of stcok in {0} at yahoo finance".format(s))
    main(stock_list=s)

This code only needs for offilne test.

Conclusion

Let us know how to download fincancial news.

I have been steuding to make model for stock using many startegy curruntly.

If you want to use this code, I’m very sorry that you should change code and make directory for data a little bit.

I should appreciate and refer for many blog on google. Thanks a lot.

If you satisfied this post you should check Github and please Star :)

Relative strength index (RSI)

2021-06-26T00:00:00+00:00

Relative Strength index (RSI)

It’s famouse for secondary indicator of stock’s market. This is presented by variance for changing upper and down.

It has tendency if it decrease, it is creasing and also if it crease, it is decresing.

Recently it should have been not true surely. However, it has been importnat index for advance afterwards.

Class code

Anyway, this is presented how to calculate the RSI.

# classRSI.py
 
import numpy as np
import pandas as pd
import pandas_datareader as pdr

class Stocks:
    def __init__(self, symbol, start_day, end_day):
        self.symbol = symbol
        self.start_day = start_day
        self.end_day = end_day

    def calcRSI(self, period=14):
        df = pdr.DataReader(self.symbol, 'yahoo',self.start_day, self.end_day)
        date_index = df.index.astype('str')
        U = np.where(df.diff(1)['Adj Close'] > 0, df.diff(1)['Adj Close'], 0)
        D = np.where(df.diff(1)['Adj Close'] < 0, df.diff(1)['Adj Close'] * (-1), 0)
        AU = pd.DataFrame(U, index=date_index).rolling(window=period, min_periods=1).mean()
        AD = pd.DataFrame(D, index=date_index).rolling(window=period, min_periods=1).mean()
        RSI = AU/ (AD+AU) * 100

        df.insert(len(df.columns), 'RSI', RSI)
        df.insert(len(df.columns), 'RSI signal', df['RSI'].rolling(window=9, min_periods=1).mean())

        return df

RSI needs variance of upper and down with average of this. And It is calculated

RSI = Average of Upper / (Average of Upper + Average of Down) * 100

Caculating RSI signal which is calculated moving avereage for RSI, it’s comparing with two indexs.

Main code of plotting

 # PlotRSI.py
  
 import numpy as np
 import pandas as pd
 import pandas_datareader as pdr
 import datetime
 import matplotlib.pyplot as plt
 import matplotlib.ticker as ticker

 from classRSI import Stocks

 def main():
     td_1y = datetime.timedelta(weeks=52/2)
     today = datetime.datetime.now()
     start_day = today - td_1y

     symbol = input('Write ticker name like aapl: ') #^IXIC

     stock = Stocks(symbol, start_day, today)

     df= stock.calcRSI()

     index = df.index.astype('str')

     fig = plt.figure(figsize=(10,10))

     ax_main = plt.subplot(1,1,1)

     def x_date(x,pos):
         try:
             return index[int(x-0.5)][:7]
         except IndexError:
             return ''

     # For figure
     ax_main.xaxis.set_major_locator(ticker.MaxNLocator(10))
     ax_main.xaxis.set_major_formatter(ticker.FuncFormatter(x_date))

     ax_main.plot(index, df['RSI'], label='RSI')
     ax_main.plot(index, df['RSI signal'], label='RSI signal')

     plt.grid()
     plt.show()

 if __name__ == "__main__":
     main()

It is simply calculating for start day to today and plot

Example figure

Typical ticker is AAPL(Apple). This figure is presented at 2021-06-26.

you just know that this plot what is mean :)

Model, Selection

For using RSI, we can make model and select ticker in index (Dow, S&P500 etc) ?

I have simple idea to use RSI value of down to up and up to down.

It is true that RSI value is under 30?

In my case, I have choose selected data by Bollinger band’s model. This model is explained this post which is also important finance index.

 # modelRSI.py
  
 import numpy as np
 import pandas as pd
 import datetime
 from tqdm import tqdm

 from classRSI import Stocks

 def main(stock_list, day_init=datetime.datetime(2020,1,1), today=datetime.datetime.now()):
     print(stock_list)

     selected_ticker = []
     for ticker in tqdm(stock_list):
         stock = Stocks(ticker, start_day, today)
         df = stock.calcRSI()
         df_recent = df.iloc[-1:]
         value_RSI = float(df_recent['RSI'])
         value_RSI_signal = float(df_recent['RSI signal'])
         down = 40
         if value_RSI < down:
             if value_RSI < value_RSI_signal:
                 selected_ticker.append(ticker)

     print(selected_ticker)

     url = '/Users/hanseopark/Work/stock/data_ForTrading/selected_ticker.json'
     df = pd.DataFrame(selected_ticker, columns=['Ticker'])
     df.to_json(url)

 if __name__ == '__main__':
     td_1y = datetime.timedelta(weeks=52/2)
     today = datetime.datetime.now()
     start_day = today - td_1y

     url = '/Users/hanseopark/Work/stock/data_ForTrading/{0}_TickerList.json'.format(today.date())
     s_list = pd.read_json(url)['Ticker'].values.tolist()
     main(stock_list = s_list, day_init = start_day)

If you change var of down like 30 to 40, you should get many ticker beyond hard condition.

Conclusion

Let us know what is RSI and how to caclulate with figure. And then we are going to find ticker using this model :)

If you want to use this code, I’m very sorry that you should change code and make directory for data a little bit.

I should appreciate and refer for many blog on google. Thanks a lot.

If you satisfied this post you should check Github and please Star :)

Bollinger band with python

2021-06-25T00:00:00+00:00

Bollinger Band (BB)

The bollinnger band is mening of standard deviation with calculation of moving average during 20, 60, and 120.

It has been known to have tendecy narrow to wide and wide to narrow again.

Additionally, If price of stock is over the line by BB, It has tendecy to be upper the line and If this is under the linne by BB, It has tendecu to be lower.

This is presented with price of stock and moving average line with Python.

It is needed to be downlaod on the web site. We have very useful method using pandas_datareader API.

pip install pandas-datareader

Class code

Anyway, the code of class is below.

# classBB.py
import pandas as pd
import pandas_datareader as pdr
import datetime

class Stocks:
		 def __init__(self, symbol, start_day, end_day, url = '', Offline=False):
         self.symbol = symbol
         self.start_day = start_day
         self.end_day = end_day
         self.url = url
         self.Offline = Offline

     def get_price_data(self):
         if (self.Offline == True):
             url_price = self.url+'/FS_{0}_Value.json'.format('sp500') # If you have nasdaq stocks, you'd like to choose  nasdaq stocks rather than sp500 or dow to have many data sets
             combined_price = pd.read_json(url_price)
             df = combined_price[combined_price.Ticker.str.contains(self.symbol)]
             df_price = df.copy()
             df_price = df_price.set_index('Date')
             df_price = df_price.drop(['Ticker'], axis=1)
             df_price = df_price.loc[self.start_day : self.end_day]

         else:
             df_price = pdr.DataReader(self.symbol, 'yahoo',self.start_day, self.end_day)

         return df_price

     def with_moving_ave(self):
         df_price = self.get_price_data()
         ma5= df_price['Adj Close'].rolling(window = 5, min_periods=1).mean()
         ma20= df_price['Adj Close'].rolling(window = 20, min_periods=1).mean()
         ma60= df_price['Adj Close'].rolling(window = 60, min_periods=1).mean()
         ma120= df_price['Adj Close'].rolling(window = 120, min_periods=1).mean()
         std = df_price['Adj Close'].rolling(window = 20, min_periods=1).std()
         bol_upper = ma20 + 2*std
         bol_down = ma20 - 2*std

         df_price.insert(len(df_price.columns), 'MA5', ma5)
         df_price.insert(len(df_price.columns), 'MA20', ma20)
         df_price.insert(len(df_price.columns), 'MA60', ma60)
         df_price.insert(len(df_price.columns), 'MA120', ma120)
         df_price.insert(len(df_price.columns), 'Std', std)
         df_price.insert(len(df_price.columns), 'bol_upper', bol_upper)
         df_price.insert(len(df_price.columns), 'bol_down', bol_down)

         return df_price

The function get_price_data() is meaning to be download for online by yahoo finance for real time and another method to be analyzied for offline by local after download the same method. It is explaind later :)

Main Code of Ploting

# PlotBB.py
 
import pandas_datareader as pdr
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
from matplotlib import gridspec
from mplfinance.original_flavor import candlestick2_ohlc

import datetime

from classBB import Stocks

def main():
    td_1y = datetime.timedelta(weeks=52/2)
    #start_day = datetime.datetime(2019,1,1)
    #end_day = datetime.datetime(2021,5,1)
    today = datetime.datetime.now()
    start_day = today - td_1y

    # To test just a ticker for making class
    symbol = input('Write ticker name like AAPL: ')
    url_data = '/Users/hanseopark/Work/stock/data_origin/'

    offline_test = False
    stock = Stocks(symbol, start_day, today, url=url_data, Offline= offline_test)

    ## For figure
    df = stock.with_moving_ave()
    index = df.index.astype('str')
    fig = plt.figure(figsize=(10,10))

    gs = gridspec.GridSpec(2,1,height_ratios=[3,1])
    ax_main = plt.subplot(gs[0])
    ax_1 = plt.subplot(gs[1])

    def x_date(x,pos):
        try:
            return index[int(x-0.5)][:7]
        except IndexError:
            return ''
    # ax_main
    ax_main.xaxis.set_major_locator(ticker.MaxNLocator(10))
    ax_main.xaxis.set_major_formatter(ticker.FuncFormatter(x_date))

    ax_main.set_title(symbol+' stock', fontsize=22 )
    #ax_main.set_xlabel('Date')
    ax_main.plot(index, df['MA5'], label='MA5')
    ax_main.plot(index, df['MA20'], label='MA20')
    ax_main.plot(index, df['MA60'], label='MA60')
    ax_main.plot(index, df['MA120'], label='MA120')
    ax_main.plot(index, df['bol_upper'], label='bol_upper')
    ax_main.plot(index, df['bol_down'], label='bol_down')
    ax_main.fill_between(index, df['bol_down'], df['bol_upper'], color = 'gray')

    candlestick2_ohlc(ax_main,df['Open'],df['High'],df['Low'],df['Close'], width=0.5, colorup='r', colordown='b')

    ax_main.legend(loc=2)

    # ax_1
    ax_1.xaxis.set_major_locator(ticker.MaxNLocator(10))
    ax_1.xaxis.set_major_formatter(ticker.FuncFormatter(x_date))
    ax_1.set_xlabel('Date')
    ax_1.set_ylabel('Std')
    ax_1.plot(index, df['Std'], label='Std')
    mean = df['Std'].mean()
    ax_1.hlines(y=mean, xmin=index[0], xmax=index[-1], colors='red')

    plt.grid()
    plt.show()

if __name__ == '__main__':
     main()

To be choice from start day to end day and if it checks offline test or not

For figure, using gridspec to be seperated canvas and candlestick2_ohlc that has advatage to make candlestick plot easily.

Any other plot library is easily explaind by google :) just search.

Example figure

Typical ticker is AAPL(Apple). upper figure is presented price with BB and down figure is presented by only standard deviation at 2021-06-25.

Model, Selection

For using Bolliner band, we can make model and select ticker in index (Dow, S&P500 etc)?

I have simple idea to use standard deviation with recent price and prior to price.

Is it true currunt standard deviation over mean of standard deviation during period?
It is true recent price below upper line of BB?
Is it true this ticker is rising?
- price of today over price of yesterday
- price of yesterday over price of the day befor yesterday

If upper the condition is satisfied, I think that selected ticker has high probabily of rising to be future.

In my case I make the code for modeling and selection below.

To download index using yahoo_fin API. This is very useful to download price, financial statements and others.

pip install yahoo-fin

# modelBB.py
import pandas as pd
import yahoo_fin.stock_info as yfs

import datetime
from tqdm import tqdm

from classBB import Stocks

def main(stock_list, day_init=datetime.datetime(2020,1,1), today=datetime.datetime.now()):

   # Get list of Dow tickers
    dow_list = yfs.tickers_dow()
    filename = ''
    if stock_list == 'dow':
        dow_list = yfs.tickers_dow()
        filename = 'dow'
    elif stock_list == 'sp500':
        filename = 'sp500'
        dow_list = yfs.tickers_sp500()
    elif stock_list == 'nasdaq':
        filename = 'nasdaq'
        dow_list = yfs.tickers_nasdaq()
    elif stock_list == 'other':
        filename = 'other'
        dow_list = yfs.tickers_other()
    print(dow_list)

    selected_ticker = []
    for ticker in tqdm(dow_list):

        stock = Stocks(ticker, start_day, today)
        df = stock.with_moving_ave()
        index = df.index.astype('str')
        mean = df['Std'].mean()
        df_day_ago_ago = df.iloc[-3]
        df_day_ago = df.iloc[-2]
        df_recent = df.iloc[-1:]
        #print(df_day_ago)
        #print(df_recent)
        value_day_ago_ago = float(df_day_ago_ago['Adj Close'])
        value_day_ago = float(df_day_ago['Adj Close'])
        value_recent = float(df_recent['Adj Close'])
        upper_recent = float(df_recent['bol_upper'])
        down_recent = float(df_recent['bol_down'])
        std_recent = float(df_recent['Std'])

        if std_recent < mean and value_recent > value_day_ago and value_day_ago > value_day_ago_ago and value_recent <   upper_recent:
            selected_ticker.append(ticker)

    url = '/Users/hanseopark/Work/stock/data_ForTrading/{0}_TickerList'.format(today.date())
    df = pd.DataFrame(selected_ticker, columns=['Ticker'])
    #df.columns = ['Ticker']
    df.to_json(url+'json')
    df.to_csv(url+'csv')

    print(df)
    return df

if __name__ == '__main__':
    s_list= input("Choice of stock's list (dow, sp500, nasdaq, other): ")
    td_1y = datetime.timedelta(weeks=52/2)
    #start_day = datetime.datetime(2019,1,1)
    today = datetime.datetime.now()
    start_day = today - td_1y

    main(stock_list = s_list,day_init = start_day)

Conclusion

Let us know what is BB (Bollinger Band) and how to calculate with figure. And then we are going to find ticekr using this model.

If you want to use this code, I’m very sorry that you should change code and make directory for data a little bit.

I should appreciate and refer for many blog on google. Thanks a lot.

If you satisfied this post you should check Github and please Star :)