KGC


[ English ]
インターネット計測とデータ解析 (GIGA/GI)

お知らせ


科目名インターネット計測とデータ解析 (GIGA/GI) [ シラバス ]
http://web.sfc.keio.ac.jp/~kjc/classes/sfc2016s-measurement/

It becomes possible to access a huge amount of diverse data through the Internt. It allows us to obtain new knowledge and create new services, leading to an innovation called "Big Data" or "Collective Intelligence". In order to understand such data and use it as a tool, one needs to have a good understanding of the technical background in statistics, machine learning, and computer network systems.
In this class, you will learn about the overview of large-scale data analysis on the Internet, and basic skills to obtain new knowledge from massive information for the forthcoming information society.


担当者 長 健二朗
授業期間2016年春学期 月曜日4時限
授業レベル 学部
参考文献 参考文献リスト


授業ビデオとマテリアルFlash Player
授業ビデオの再生にはFlash Playerが必要です。
受講したい回をクリックしてください。
第01回2016/04/11 Introduction
- exercise script (count.rb)
- exercise script (count-rubyish.rb)
- optional reading material (URL)
- lecture slides (pdf)
- Big Data and Collective Intelligence
- Internet measurement
- Large-scale data analysis
- exercise: introduction of Ruby scripting language


第02回2016/04/18 Data and variability
- lecture slides (pdf)
- exercise data (marathon.txt)
- exercise script (mean.rb)
- exercise script (stddev.rb)
- exercise script (stddev2.rb)
- exercise script (median.rb)
- exercise script (marathon.plt)
- exercise script (marathon-cdf.rb)
- exercise script (marathon-cdf.plt)
- Summary statistics
-  Sampling
-  How to make good graphs
- exercise: graph plotting by Gnuplot


第03回2016/04/25 Data recording and log analysis
- lecture slides (pdf)
- exercise data (sample_access_log.zip) (21MB)
- exercise script (parse_accesslog.rb)
- exercise script (access.plt)
- Network management tools
- Data format
- Log analysis methods
- exercise: log data and regular expression


第04回2016/05/02 Distribution and confidence intervals
- lecture slides (pdf)
- exercise script (box-muller.rb)
- exercise script (box-muller-hist.rb)
- exercise script (box-muller-hist.plt)
- exercise script (conf-interval.rb)
- exercise script (conf-interval.plt)
- Normal distribution
- Confidence intervals and statistical tests
- Distribution generation
- exercise: confidence intervals
- assignment 1


第05回2016/05/09 Diversity and complexity
- lecture slides (pdf)
- exercise data (us-surnames.csv)
- exercise script (make_ccdf.rb)
- exercise script (ccdf.plt)
- exercise script (count_contents.rb)
- Long tail
- Web access and content distribution
- Power-law and complex systems
- exercise: power-law analysis


第06回2016/05/16 Correlation
- lecture slides (pdf)
- exercise script (correlation.rb)
- exercise data (correlation-data-1.txt)
- exercise data (correlation-data-2.txt)
- exercise script (similarity.rb)
- exercise data (scores.txt)
- Online recomendation systems
- Distance
- Correlation coefficient
- exercise: correlation analysis


第07回2016/05/23 Multivariate analysis
- lecture slides (pdf)
- exercise script (leastsquare.rb)
- exercise script (pca.rb)
- exercise data (pca-data.txt)
- Data sensing and GeoLocation
- Linear regression
- Principal Component Analysis
- exercise: linear regression


第08回2016/06/06 Time-series analysis
- exercise data (autocorr_5min_data.txt)
- exercise data (ifbps-201205.txt)
- exercise script (autocorr.rb)
- exercise script (autocorr.plt)
- exercise script (hourly_out.rb)
- exercise script (hourly_out.plt)
- exercise script (week_out.rb)
- exercise script (week_out.plt)
- exercise script (correlation_out.rb)
- lecture slides (pdf)
- Internet and time
- Network Time Protocol
- Time series analysis
- exercise: time-series analysis
- assignment 2


第09回2016/06/13 Topology and graph
- lecture slides (pdf)
- exercise script (dijkstra.rb)
- exercise data (topology.txt)
- exercise data (jr.txt)
- Routing protocols
- Graph theory
- exercise: shortest-path algorithm


第10回2016/06/20 Anomaly detection and machine learning
- lecture slides (pdf)
- exercise script (naivebayes.rb)
- Anomaly detection
- Machine Learning
- SPAM filtering and Bayes theorem
- exercise: naive Bayesian filter


第11回2016/06/27 Data Mining
- lecture slides (pdf)
- exercise script (k-means.rb)
- exercise script (km-results.plt)
- exercise data (km-data.txt)
- Pattern extraction
- Classification
- Clustering
- exercise: clustering


第12回2016/07/04 Search and Ranking
- lecture slides (pdf)
- exercise script (pagerank.rb)
- exercise data (sample-links.txt)
- exercise data (links-100k.txt) (URL)
- Search systems
- PageRank
- exercise: PageRank algorithm


第13回2016/07/11 Scalable measurement and analysis
- lecture slides (pdf)
- exercise script (wc-map.rb)
- exercise script (wc-reduce.rb)
- exercise data (wc-data.txt)
- Distributed parallel processing
- Cloud computing technology
- MapReduce
- exercise: MapReduce algorithm



Powered by SOI Copyright(c) 2002-2019, Keio University Shonan Fujisawa Campus. All rights reserved.
このサイトの著作権について