Contents

• File Structure
• Detailed Information of the Files
• How to Run the Code
• Visualization System

File Structure

Visualization-Final-Project
├──js                   # static .js files
    ├──d3.v5.min.js         # d3.js version 5, used in main.html
    ├──d3.v6.js             # d3.js version 6, used in relationship.html
    └──topojson.js          # draw map
├──output_csv           # csvs containing the information of given data
    ├──area.csv             # historical information of areaid in China
    ├──ID_NAME_convert.csv  # converter between the area-id and area-name
    ├──adult.csv            # information of the illegal adults
    ├──minor.csv            # information of the illegal minors
    ├──bar.csv              # information of the illegal bars
    ├──float.csv            # information of the migrant population
    └──all_info.csv         # information of bars' profile
├──pycode               # all the python codes for the manipulation of data
    ├──__init__.py  
    ├──Problem1.py          # python code for problem 1, following are the same
    ├──Problem2.py          # ...
    ├──Problem3.py          # ...
    ├──Problem4.py          # ...
    ├──main.py              # process data from problem 1 to 4
    ├──utils.py             # contains all auxiliary functions
    └──requirements.text    # package requirements of these python codes
├──templates            
    ├──main.html            # visualization system
    ├──relationship.html    # the sub-page of relationship visualization 
    ├──geo.json             # map data of chongqing in geojson format     
    └──weighted_graph.json  # relationship between people 
├──img                  # some images used in readme or html
    ├──home.png             # home icon in relationship.html
    ├──full.png
    ├──community.png
    ├──output.png
    └──map.png
├──others
    ├──report.pdf           # report of the project in pdf format
    ├──report.docx          # report of the project in docx format
    ├──demo video.mp4       # demo video
    └──ppt.pptx             # ppt of the presentation
└──readme.md            # readme

Detailed Information of the Files

• output_csv
  • area.csv/ID_NAME_convert.csv
    • The data of area.csv is downloaded from here.
    • A converter between area-id and area-name is generated by finding the first nonempty column(year) in each row.
  • adult.csv/minor.csv/bar.csv
    • These csv files store the answer for Problem 1.
    • The columns of the adult and minor are [id, name, times], with the meaning of personal-id, name, illegal-times respectively.
    • The columns of the bar also include [adult, minor, lng, lat], which stand for adult-illegal-times, minor-illegal-times, longitude, latitude.
  • float.csv
    • Contains [area, hour, duration, age, xb]
  • all_info.csv
    • Contains [siteid, xb, duration, area, age, hour]
• pycode
  • Refer to the comment in section file structure.
• templates
  • main.html
    • Contains five separate divs: map, board, time, age, people. They are used for displaying map, leaderboard, curve line, bar plot and pie respectively.
  • relationship.html
    • Contains one div, displaying the relationship between people.
  • geo.json
    • The data of Chongqing's map in GeoJSON format. Downloaded from here.
  • weighted_graph.json
    • Relationship between people with weight information. Generated by the sliding window method.

How to Run the Code

• Step 1
  • Open Visualization-Final-Project and set it as the working directory.
• Step 2 (optional)
  • Create a data file in the working directory and save swjl and wb data in this file. The original dataset can be downloaded from the following two links: Challenge 2.1, Challenge 2.2.
  • Directly run main.py in pycode.
  • Or you can type python ./pycode/main.py in terminal.
  • The output in terminal should be like this:

  Note:

  • The cleaning and process of swjl and wb data may take several minutes, you can refer to the output in terminal for the information of execution.
  • Since the processed information(only contains the information of the first four swjl csv files) is already contained in file output_csv, you're free to jump to Step 3.

• Step 3
  • Open main.html in templates.
  • Right click and choose Open with Live Server.

  Note:

  • There are about $15\text{M}$ rows of data(if you use the full data set), so the initialization of the visulation system may take more than 30s. Some patience is needed here.
  • For the purpose of presentation, I have selectively utilized four csv files as the foundational dataset.
  • The extension Live Server is required here.
  • The port of your local server should be 5500.

Visualization System

• Snap shot(Full data set)
• Demo video