Data Science Methodology (from Coursera)

- From problem to approach -

Q1. What is the problem that you are trying to solve?

Q2. How can you use data to answer the question?

 

- Working with the data -

Q3. What data do you need to answer the question?

Q4. Where is the data coming from (identify all sources) and how will you get it? 

Q5. Is the data that you collected representative of the problem to be solved?

Q6. What additional work is required to manipulate and work with the data?

 

- Deriving the answer -

Q7. In what way can the data be visualized to get to the answer that is required?

Q8. Does the model used really answer the initial question or does it need to be adjusted?

Q9. Can you put the model into practice?

Q10. Can you get constructive feedback into answering the question?

---

cf. Data Mining - CRISP-DM Methodology

* Intent: to take case specific scenarios and general behaviors to make them domain neutral

 

 

→ Business Understanding) This stage is the most important because this is where the intention of the project is outlined. It requires communication and clarity. The difficulty here is that stakeholders have different objectives, biases, and modalities of relating information. They don’t all see the same things or in the same manner. Without clear, concise, and complete perspective of what the project goals are resources will be needlessly expended.

→ Data Understanding) Data understanding relies on business understanding. Data is collected at this stage of the process. The understanding of what the business wants and needs will determine what data is collected, from what sources, and by what methods.

→ Data Preparation) Once the data has been collected, it must be transformed into a useable subset unless it is determined that more data is needed. Once a dataset is chosen, it must then be checked for questionable, missing, or ambiguous cases.

→ Modeling) Once prepared for use, the data must be expressed through whatever appropriate models, give meaningful insights, and hopefully new knowledge. This is the purpose of data mining: to create knowledge information that has meaning and utility. The use of models reveals patterns and structures within the data that provide insight into the features of interest. Models are selected on a portion of the data and adjustments are made if necessary. Model selection is an art and science.

→ Evaluation) The selected model must be tested. This is usually done by having a pre-selected test, set to run the trained model on. This will allow you to see the effectiveness of the model on a set it sees as new. Results from this are used to determine efficacy of the model and foreshadows its role in the next and final stage

→ Deployment) In the deployment step, the model is used on new data outside of the scope of the dataset and by new stakeholders. The new interactions at this phase might reveal the new variables and needs for the dataset and model.

※ The key point of this process is that it’s cyclical; therefore, even at the finish you are having another business understanding encounter to discuss the viability after deployment. The journey continues.

 

 

1) From Problem to Approach

# Business Understanding: What is the problem that you are trying to solve?

- Understanding the GOAL of the person who is asking the question

- Figure out the objectives that are in support of the goal

- Depending on the problem, different stakeholders will need to be engaged in the discussion to help determine requirements and clarify questions

 

# Analytic Approach: How can you use data to answer the question?

- The correct approach depends on business requirements for the model

e.g)

- if the question is to determine probabilities of an action) a predictive model

- if the questions is to show relationships) a descriptive model (where clusters of similar activities based on events and preferences are examined)

- if the question requries a yes/no answer) a classification model

→ categorical outcome & explicit 'decision path' showing conditions leading to high risk & likelihood of classified outcome, easy to understand & apply

e.g) A data scientist determines that building a recommender system is the solution for a particular business problem at hand

2) From Requirements to Collection

# Data Requirements: What are Data Requirements?

- involves identifying the necessary data content, formats and sources for initial data collection

 

# Data Collection: What occurs during Data Collection?

- Once the data ingredients are collected, then in the data collection stage, the data scientist will have a good understanding of what they will be working with. 

- Techniques such as descriptive statistics and visualization can be applied to the data set, to assess the content, quality, and initial insights about the data. 

- Gaps in data will be identified and plans to either fill or make substitutions will have to be made

- It is alright to defer decisions about unavailable data(data requirements are revised), and attempt to acquire it at a later stage.

- merging data - DBAs and programmers often work together to extract data from various sources, and then merge it. This allows for removing redundant data, making it available for the next stage of the methodology, which is data understanding. At this stage, if necessary, data scientists and analytics team members can discuss various ways to better manage their data, including automating certain processes in the database, so that 

data collection is easier and faster.

3) From Understanding to Preparation

# Data Understanding: encompasses all activities related to constructing the data set.

- Is the data that you collected representative of the problem to be solved?

- descriptive statistics needed to be run against the data columns that would become variables in the model.

e.g)

- these statistics included Nearst, univariates, and statistics on each variable, such as mean, 

median, minimum, maximum, and standard deviation

- pairwise correlations were used, to see how closely certain variables were related, and which ones, if any, were very highly correlated, meaning that they would be essentially redundant, thus making only one relevant for modeling.

- histograms of the variables were examined to understand their distributions.

(a good way to understand how values or a variable are distributed, and which sorts of data preparation may be needed to make the variable more useful in a model)

- are also used to access data quality) from the information provided, certain values can be re-coded or perhaps even dropped if necessary, such as when a certain variable has missing values. The question then becomes, does "missing" mean anything? Sometimes a missing value might mean "no", or "0" (zero), or at other times it simply 

means "we don't know". Or, if a variable contains invalid or misleading values, such as a numeric variable called "age" that contains 0 to 100 and also 999, where that "triple-9" actually means "missing", but would be treated as a valid value unless we corrected it.

- can loop back to the data collection stage and build a more comprehensive(or narrow) definition of the keyword (interactive processes in the methodology)

→ The more one works with the problem and the data, the more one learns and therefore the more refinement that can be done within the model, ultimately leading to a better solution to the problem.

 

# Data Preparation: the most time-consuming phase of a data science project, typically taking seventy percent and even up to even ninety percent of the overall project time.

- transforming data in the data preparation phase is the process of getting the data into a state where it may be easier to work with

- (1) define the variables to be used in the model / (2) determine the timing of events / (3) aggregate the data and merge them from different sources / (4) identify missing data

- it must be prepared in a way that addresses missing or invalid values and removes duplicates, toward ensuring that everything is properly formatted (including addressing outliers)

- can be accelerated through automation

- feature engineering = the process of using domain knowledge of the data to create features that make the machine learning algorithms work (critical when ML tools are being applied to analyze the data)

(A feature is a characteristic that might help when solving a problem. Features within the data are important to predictive models and will influence the results you want to achieve) 

4) From Modeling to Evaluation

# Data Modeling: In what way can the data be visualized to get to the answer that is required?

- focuses on developing models that are either descriptive or predictive

 

 

* descriptive model) if a person did this, then they're likely to prefer that

* predictive model) tries to yield yes/no, stop/go outcomes

 

→ The data scientist will use a training set for predictive modelling

→ A training set is a set of historical data in which the outcomes are already known. The training set acts like a gauge to determine if the model needs to be calibrated. In this stage, the data scientist will play around with different algorithms to ensure that the variables in play are actually required

→ The end goal is to move the data scientist to a point where a data model can be built to answer the question

 

# Data Evaluation: Does the model used really answer the initial question or does it need to be adjusted?

* includes ensuring that the data are properly handled and interpreted

 

(1) Diagnostic Measures Phase - to ensure the model is working as intended

→ If the model is a predictive model, a decision tree can be used to evaluate if the answer the model can output, is aligned to the initial design. It can be used to see where there are areas that require adjustments. 

→ If the model is a descriptive model, one in which relationships are being assessed, then a testing set with known outcomes can be applied, and the model can be refined as needed

 

(2) Statistical Significane Testing - to ensure that the data is being properly handled and interpreted within the model

→ This is designed to avoid unnecessary second guessing when the answer is revealed.

e.g.) ROC curve (a useful diagnostic tool for determining the optimal classification model)

 

* modeling and evaluation are iterative processes

5) From Deployment to Feedback

# Data Deployment

- Depending on the purpose of the model, it may be rolled out to a limited group of users or in a test environment, to build up confidence in applying the outcome for use across the board.

- Understand the results -> gathering application requirements -> additional requirements

 

# Data Feedback

- feedback from the users will help to refine the model and assess it for performance and impact. 

- The value of the model will be dependent on successfully incorporating feedback and making adjustments for as long as the solution is required

- Once the model is evaluated and the data scientist is confident it'll work, it is deployed and put to the ultimate test: actual, real-time use in the field

- feedback -> refining the model -> redeployment

 

* these stages are highly ITERATIVE!

---

* 출처1) ≪Data Science Methodology≫ by Coursera 🧡

* 출처2) www.educba.com/predictive-analytics-vs-descriptive-analytics/