Dealing with imbalanced data in R programming

This is a good article.

 https://www.r-bloggers.com/2019/04/methods-for-dealing-with-imbalanced-data/

Unfortunately R glm does not have the class_weight variable directly. A little manipulation is needed to create the weights vector. 

How to deal with unbalanced dataset in Machine Learning (ML)

 Unbalanced dataset is very common. For example, credit card transaction (majority of them are authentic), malware detection ( majority are benign), internet traffic( majority are friendly), CT-scan ( majority without tumor), etc.

Why we need to deal with it and how to deal with it. Here we are going to use Jupyter notebook to illustrate this problem. 

I am writing this post little by little, so it may takes a few days to finish. 

https://github.com/chaowu2009/ML_Projects/blob/master/ML_unbalanced_data.ipynb

pandas, groupby result to file

 grpd = df.groupby('name').size()

print(grpd)

grpd.reset_index().to_csv('result.csv')

Modern portfolio theory (MPT)

https://www.quantconnect.com/tutorials/introduction-to-financial-python/modern-portfolio-theory

https://www.quantconnect.com/tutorials/introduction-to-financial-python/modern-portfolio-theory

Modern portfolio theory model implementation in Python

 

https://medium.com/analytics-vidhya/modern-portfolio-theory-model-implementation-in-python-e416facabf46

Portfolio Optimization using MPT in Python

https://www.analyticsvidhya.com/blog/2021/04/portfolio-optimization-using-mpt-in-python/

pandas-datareader error

Uninstall them

#!pip uninstall yfinance

#!pip uninstall pandas-datareader

Reinstall them

#!pip install yfinance --upgrade --no-cache-dir

#!pip install pandas-datareader

 

# If you do it in the jupyter notebook, it may not work

# Do them in command line.

 

DODD-FRANK ACT STRESS TESTS (DFAST)

Dodd-Frank Act stress testing is a forward-looking exercise that assesses the impact on capital levels that would result from immediate financial shocks and nine quarters of severely adverse economic conditions.  FHFA requires Fannie Mae and Freddie Mac to submit the results of stress tests based on two scenarios: a Baseline scenario and a Severely Adverse scenario.  FHFA aligned the stress test scenario variables and assumptions with those used by the Board of Governors of the Federal Reserve System in its annual Dodd-Frank Act stress tests.  As of March 2020, according to the Stress Testing of Regulated Entities Final Rule, the Federal Home Loan Banks are no longer required to conduct Dodd-Frank Act stress tests.  As a prudential matter, FHFA expects the FHLBanks to continue to perform other stress tests (addressing market, credit, liquidity, and model risks) as outlined in FHFA's regulations and guidance.  

In 2014 FHFA began requiring its regulated entities  to conduct stress tests pursuant to the Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 (Dodd-Frank Act).  As amended by Section 401 of the Economic Growth, Regulatory Relief, and Consumer Protection Act (EGRRCPA), the Dodd-Frank Act requires certain financial companies with total consolidated assets of more than $250 billion, and which are regulated by a primary federal financial regulatory agency, to conduct periodic stress tests to determine whether the companies have sufficient capital to absorb losses and support operations during adverse economic conditions.

https://www.fhfa.gov/SupervisionRegulation/DoddFrankActStressTests

Risk Appetite Statement by various companies

 JP Morgan Chase (2020 annual report)

Risk appetite The Firm’s overall appetite for risk is governed by a “Risk Appetite” framework. The framework and the Firm’s risk appetite are set and approved by the Firm’s CEO, Chief Financial Officer (“CFO”) and CRO. Quantitative parameters and qualitative factors are used to monitor and measure the Firm’s capacity to take risk consistent with its stated risk appetite. Qualitative factors have been established to assess select operational risks, and impact to the Firm’s reputation. Risk Appetite results are reported to the Board Risk Committee.