How to deal with unbalanced dataset in Machine Learning (ML)

 Unbalanced dataset is very common. For example, credit card transaction (majority of them are authentic), malware detection ( majority are benign), internet traffic( majority are friendly), CT-scan ( majority without tumor), etc.

Why we need to deal with it and how to deal with it. Here we are going to use Jupyter notebook to illustrate this problem. 

I am writing this post little by little, so it may takes a few days to finish. 

https://github.com/chaowu2009/ML_Projects/blob/master/ML_unbalanced_data.ipynb

Mac OS Log Analysis (1)

Recently, my Mac laptop's (Majove, MacBook Pro (15-inch, 2017)) Safari failed to open. I searched around and could not find a solution. Even our company's helpdesk could not find a solution. Then I am curious to examine the log and try to figure it why the Safari launch failure happened.

Right now, the Safari still does not work. I solely use Chrome for web browsing.

Meanwhile, I am doing some research on cyber security. There are many works on examining on Windows or Linux logs and not so many focuses on the MacOS.

I examined the OSX Collecto (https://github.com/Yelp/osxcollector/blob/master/osxcollector/osxcollector.py) and feel that is not what I want. I begin to write some tools to analyze MacOS logs.

There is another post on this topic too. http://macadmins.psu.edu/wp-content/uploads/sites/24696/2016/06/psumac2016-19-osxlogs_macadmins_2016.pdf

Here I am summarizing those analysis step by step. I wish those will help our readers.

adversarial machine learning (1)

This series are to help readers to get familiar with different adversarial attacks in the adversarial machine learning domain.

We are going through different approaches introduced in https://adversarial-robustness-toolbox.readthedocs.io/en/latest/index.html