Using ack and regex to convert unstructured NASA GISS temperature data to CSV

by dann
Download
.json - original recording .txt - plain text version .gif - animated GIF
macOS • xterm-256color • bash 3553 views

Simple exercise in text processing/wrangling with a grep-like (ack) and regular expressions.

Context: http://2017.compciv.org/syllabus/assignments/homework/contrived-cli-data-crunching.html#nasa-annual-global-temperature-averages

Original data URL: https://data.giss.nasa.gov/gistemp/graphs_v3/Fig.A.txt

Test data URL: http://stash.compciv.org/2017/nasa-gistemp-sample.txt

Ack: beyondgrep.com/documentation/ack-2.14-man.html

Regex basics:

Note that I throw in an extra ack for clarity sake, to separate the captured-group matching from the pattern matching. To do everything with one ack:

   curl -s http://stash.compciv.org/2017/nasa-gistemp-sample.txt  \
          |  ack '(\d{4}) +(-?\d+\.\d+)' --output '$1,$2'

More recordings by dann

Browse all

ignore this blah blah 0:30

by dann

How funny is this Obama speech? 0:42

by dann

untitled 0:05

by dann

untitled 0:19

by dann
Share this recording
https://asciinema.org/a/100135
Copied!

Append ?t=30 to start the playback at 30s, ?t=3:20 to start the playback at 3m 20s.

See sharing docs for more link customization options.

Embed as image link

Use snippets below to display a preview image linking to this recording.
Ideal for places where scripts are not allowed, such as project README files.

HTML: 

<a href="https://asciinema.org/a/100135" target="_blank"><img src="https://asciinema.org/a/100135.svg" /></a>
Copied!

Markdown: 

[![asciicast](https://asciinema.org/a/100135.svg)](https://asciinema.org/a/100135)
Copied!

Embed the player

If you're embedding on your own page or on a site which permits script tags, you can use the full player widget:

<script src="https://asciinema.org/a/100135.js" id="asciicast-100135" async="true"></script>
Copied!

Paste the above script tag where you want the player to be displayed on your page.

See embedding docs for more player customization options.

Download this recording

You can download this recording in asciicast v1 format, as a .json file.

Download

Replay in terminal

You can replay the downloaded recording in your terminal using the asciinema play command: 

asciinema play 100135.json
Copied!

If you don't have asciinema CLI installed then see installation instructions.

Use with stand-alone player on your website

Download asciinema player from the releases page (you only need .js and .css file), then use it like this: 

<!DOCTYPE html>
<html>
<head>
  <link rel="stylesheet" type="text/css" href="/assets/asciinema-player.css" />
</head>
<body>
  <div id="player"></div>
  <script src="/assets/asciinema-player.min.js"></script>
  <script>
    AsciinemaPlayer.create(
      '/assets/100135.json',
      document.getElementById('player'),
      { cols: 116, rows: 22 }
    );
  </script>
</body>
</html>

See asciinema player quick-start guide for full usage instructions.

Generate GIF from this recording

While this site doesn't provide GIF conversion at the moment, you can still do it yourself with the help of asciinema GIF generator utility - agg.

Once you have it installed, generate a GIF with the following command:

agg https://asciinema.org/a/100135 demo.gif
Copied!

Or, if you already downloaded the recording file:

agg demo.cast demo.gif
Copied!

Check agg --help for all available options. You can change font family and size, select color theme, adjust speed and more.

See agg manual for full usage instructions.