What is Data Science? (from Coursera)

1) Defining Data Science & What Data Scientists Do

* What is Data Science?

 

= the field of exploring, manipulating, and analyzing data, and using data to answer questions or make recommendations.

→ what is NEW? the vast quantity of data available from massively varied sources

 

* the process of data science

 

- many organizations use data science to focus on a specific problem, and so it's essential to clarify the question that the organization wants answered

- ask questions to clarify the business need

- "what data do we need, and where will that data come from?"

- analyze the data in different ways

- after the data has revealed its insights, communicate the results to the project stakeholders (can use powerful data visualization tools)

- changing the way organizations understand the world

 

* Advice

 

- to be curious, argumentative(hypothesis), judgemental(to have preconceived notions)

- some comfort & flexibility with analytic platforms

- the ability to tell a story

- want to be a data scientist in any fields? or in specific field? (figure out your competitive advantage - know sets of skills)

cf. competitive advantage) understanding of some aspect of life (where you exceed beyond others)

 

* Old & New Problems, Data Science Solutions

 

- all organizations ultimately use data science for the same reason—to discover optimum solutions to existing problems

- innovative solutions for old problems

cf. Uber uses data to put the right number of drivers in the right place, at the right time, for a cost the rider is willing to pay

- MUST: Identify the problem and establish a clear understanding of it. Gather the data for analysis. Identify the right tools to use, and develop a data strategy.

- structured data(something which fits nicely into tables and columns and rows) & non-structured data(a weblog)

- regression: constant, the relationship between the fare and the distance (say when you get in a taxi)

 

* Cloud for Data Science

 

- to bypass the physical limitations of the computers and the systems you're using and it allows you to deploy the analytics and storage capacities of advanced machines that do not necessarily have to be your machine or your company's machine

- allows you not just to store large amounts of data on servers somewhere in California or in Nevada, but it also allows you to deploy very advanced computing algorithms and the ability to do high-performance computing using machines that are not yours

- it allows multiple entities to work with same data at the same time

- Multiple collaborators or teams can access the data simultaneously, working together on producing a solution

2) Data Science Topics Introduction

* Big Data

 

“Big Data refers to the dynamic, large and disparate volumes of data being created by people, tools, and machines. It requires new, innovative, and scalable technology to collect, host, and analytically process the vast amount of data gathered in order to derive real-time business insights that relate to consumers, risk, profit, performance, productivity management, and enhanced shareholder value.”

 

5V's>

- velocity = the speed at which data accumulates. Data is being generated extremely fast, in a process that never stops.

- volume = the scale of the data, or the increase in the amount of data stored

- variety =  the diversity of the data. Structured data fits neatly into rows and columns, in relational databases while unstructured data is not organized in a pre-defined way, like Tweets, blog posts, pictures, numbers, and video. Data comes from different sources, machines, people, and processes, both internal and external to organizations

- veracity(정확성) = the quality and origin of data, and its conformity to facts and accuracy. Attributes include consistency, completeness, integrity, and ambiguity

- value = our ability and need to turn data into value

 

* Digital Transformation

 

- The availability of vast amounts of data, and the competitive advantage that analyzing it brings, has triggered digital transformations throughout many industries
- Digital transformation is not simply duplicating existing processes in digital form; the in-depth analysis of how the business operates helps organizations discover how to improve their processes and operations, and harness the benefits of integrating data science into their workflows

 

* Data Mining

 

- Establishing Data Mining Goals: the cost benefit trade-offs for the desired level of accuracy are important considerations for data mining goals.

- Selecting Data: identifying the right kind of data needed for data mining that could answer the questions at reasonable costs is critical.

- Preprocessing Data: identify the irrelevant attributes of data and expunge such attributes from further consideration. At the same time, identifying the erroneous aspects of the data set and flagging them as such is necesary. Data should be subject to checks to ensure integrity. Lastly, you must develop a formal method of dealing with missing data and determine whether the data are missing randomly or systematically.

- Transforming Data: the next step is to determine the appropriate format in which data must be stored. An important consideration in data mining is to reduce the number of attributes needed to explain the phenomena. Data reduction algorithms, such as Principal Component Analysis can reduce the number of attributes without a significant loss in information.

- Storing Data: The transformed data must be stored in a format that makes it conducive for data mining. Data safety & privacy

- Mining Data: This step covers data analysis methods, including parametric and non-parametric methods, and machine-learning algorithms. A good starting point for data mining is data visualization

- Evaluating Mining Results: Formal evaluation could include testing the predictive capabilities of the models on observed data to see how effective and efficient the algorithms have been in reproducing data (In-sample forecast).

 

* Deep Learning & Machine Learning & AI (with Data Science)

 

- Machine Learning: a subset of AI that uses computer algorithms to analyze data and make intelligent decisions based on what it is learned without being explicitly programmed. (Machine learning algorithms are trained with large sets of data and they learn from examples. enables machines to solve problems on their own and make accurate predictions using the provided data)

- Deep Learning:  a specialized subset of machine learning that uses layered neural networks to simulate human decision-making (Deep learning algorithms can label and categorize information and identify patterns. It is what enables AI systems to continuously learn on the job and improve the quality and accuracy of results by determining whether decisions were correct)

- (Artificial) Neural Networks (in AI): a collection of small computing units called neurons that take incoming data and learn to make decisions over time. Neural networks are often layer-deep.

- AI and Data Science: Data Science can use many of the AI techniques to derive insight from data. There is some interaction between AI and Data Science, but one is not a subset of the other. Rather, Data Science is a broad term that encompasses the entire data processing methodology while AI includes everything that allows computers to learn how to solve problems and make intelligent decisions. Both AI and Data Science can involve the use of big data.

3) Data Science in Business

* The Final Deliverable

 

- includes powerful narrative

- The initial planning and conceptualizing of the final deliverable is extremely important for producing a compelling document

 

* How Can Someone become a Data Scientist & Recruiting

 

- a real data scientist, the high-end data scientists, are mostly PhDs

- to know how to program, at least have some computational thinking, need to know some algebra, at least up to analytics, geometry, and hopefully some calculus, some basic probability, some basic statistics, 

- relational databases, computer science theory & statistics, probability

- the person who has all the skills, that they know the domain-specific knowledge. They're excellent in analyzing structured and unstructured data. And they're great at presenting and they've got great storytelling skills.

- but what really matters is who's passionate about the kind of business that you do. Someone could be a great data scientist in the retail environment, but they may not be that excited about working in IT related firms or working with gigabytes of weblogs.

- CURIOSITY, SENSE of HUMOR (look at the lighter elements), technical skills

- techinical skills)

 

1) what kind of a technical platform would you like to adopt?

- If you want to work in the traditional market research data, structure data environment, your skills should be some statistical knowledge and some knowledge of basic statistical algorithms, maybe some machine learning algorithms. / If you want to work in big data, then there's the other aspect of it and that is to be able to store data

- it starts with where you would like to be, in what field, in what domain.

 

2) to have the ability to communicate

- If you have a very large table, can you synthesize this and make it more appealing that when it goes on the screen, or is it part of a document that it just speaks? It sings the findings and the reader just gets it right there.  So the ability to present your findings, either verbally, or in a presentation, or in a document.

 

* The Report Structure

 

- the length of the reports varied depending largely on the purpose of the report. Brief reports were drafted as commentaries on current trends and developments that attracted public or media attention. Detailed and comprehensive reports offered a critical review of the subject matter with extensive data analysis and commentary

 

- cover page) at a minimum, the cover page should include the title of the report, names of authors, their affiliations, and contacts, name of the institutional publisher (if any), and the date of publication

- table of contents) an abstract or an executive summary (even for a short document)

- introductory section) helpful in setting up the problem for the reader who might be new to the topic and who might need to be gently introduced to the subject matter before being immersed in intricate details. You might use literature review to highlight gaps in the existing knowledge, which your analysis will try to fill

- methodology section) introduce the research methods and data sources you used for the analysis

- results section) present empirical findings

- discussion section) where rely on the power of narrative to enable numbers to communicate your thesis to your readers

- conclusion section) generalize your specific findings and take on a rather marketing approach to promote your findings so that the reader does not remain stuck in the caveats that you have voluntarily outlined earlier. 

- the list of references

- acknowledgement section) acknowledging the support of those who have enabled your work is always good

- appendices (if needed)

---

* 출처> 'what is data science?' by Coursera