Hadoop: Data Processing and Modelling

Time for action – correlating of sighting duration to UFO shape

Let's do a little more detailed analysis in regards to this shape data. We wondered if there was any correlation between the duration of a sighting to the reported shape. Perhaps cigar-shaped UFOs hang around longer than the rest or formations always appear for the exact amount of time.

1. Save the following to shapetimemapper.rb:

   #!/usr/bin/env ruby pattern = Regexp.new /\d* ?((min)|(sec))/ while line = gets parts = line.split("\t") if parts.size == 6 shape = parts[3].strip duration = parts[4].strip.downcase if !shape.empty? && !duration.empty? match = pattern.match(duration) time = /\d*/.match(match[0])[0] unit = match[1] time = Integer(time) time = time * 60 if unit == "min" puts shape+"\t"+time.to_s ...