golang reads and parses csv files

Exported and downloaded the CSV file of live visitors from the Baidu statistics background. I want to parse it locally and import it into MySQL for backup, because Baidu statistics can only keep two weeks of historical data, and the limit is 5000 items.

golang resolves csv code

The golang’s built-in “encoding/csv” library is used here to automatically read a row of csv data and return slice type data.

package main

import (
	"encoding/csv"
	"FMT."
	iconv "github.com/djimenez/iconv-go"
	"io."
	"log"
	"os"
)

func main() {
	// Open the file
	csvfile, err := os.Open("2020-05-21.csv")
	if err ! = nil {
		log.Fatalln("Couldn't open the csv file", err)
	}
	defer csvfile.Close()

	// Parse the file
	r := csv.NewReader(csvfile)

	// Iterate through the records
	for {
		// Read each record from csv
		record, err := r.Read()
		if err == io.EOF {
			break
		}
		if err ! = nil {
			log.Fatal(err)
		}

		fmt.Printf ("Record has %d columns.\n", len(record))
		city, _ := iconv.ConvertString(record[2], "gb2312", "utf-8")

		fmt.Printf("%s %s %s %s \n", record[0], record[1], city)
	}
}

encoding conversion

The CSV exported by Baidu Statistics is very odd, with the file encoded as latin1, causing the Chinese in it to appear as garbled.

Baidu must still be using GB2312, so it needs to convert the fields containing Chinese. This library is used here.

github.com/djimenez/iconv-go

Analysis results

Take one row of data as an example.

Record has 28 columns.
1027 2020/05/21 00:04:17 Baotou