DECO

From Sensorcast
Jump to: navigation, search

Distributed Electronic Cosmic-Ray Observatory.
Current DECO data viewable here.
Device ID list for current DECO data is here

Notes

Goal

The Distributed Electronic Cosmic Ray Observatory (DECO) is an effort to harness the global network of mobile devices (primarily devices running Android software), to detect the origin and behavior of the cosmic radiation.

History

Cosmic Rays were first discovered in 1912 by Victor Hess, who noted that radiation flux increased with altitude. An unexpected phenomenon, this was the birth of particle physics. In the years since, other particles such as the positron (1933) and the muon (1937) have been discovered, as well as general equations for mapping out the dispersal of the radiation as it approaches the earth.

Detection of cosmic-rays

On the Earth's surface cosmic-rays are not directly detected, but rather the byproducts of their interactions with Earth's atmosphere are used to reconstruct the information about the original incoming particle. Figure 1 gives a sketch of three of the established methods for detecting extensive air showers.

Figure 1. Click to view. The primary ground-based methods of detecting cosmic-rays via extensive air showers.

Animations of an extensive air shower from both an overhead and side view are available here
As seen in figure 1, the geographic extent of an extensive air shower can extend across square kilometers of the Earth's surface. This necessitates the use of large arrays of detectors in order to study the properties of an incoming cosmic-ray, such as initial energy and direction. With DECO one can use the CCDs found in cellphone cameras as an existing, and low-cost, set of detectors in order to run a cosmic-ray detection array as a supplement to existing arrays built specifically for this purpose.
The measured flux of cosmic-rays on phones running DECO are initially being calibrated against existing cosmic-ray detectors provided by Fermilab's Quarknet program. The hardware is featured in figures 2-4.

Figure 2 Figure 3 Figure 4
Figure 2. Click to view. This is a piece of hardware used for detecting and counting charged particles that hit the black wrapped scintillator panels.
Figure 3. Click to view. The electronic counter of charged particles.
Figure 4. Click to view. The black wrapped scintillator panels up close

Motivations for detection


Figure 5 Figure 6 Figure 7
Figure 5. Click to view. Charged particles colliding.
Figure 6. Click to view. High contrast photo of cosmic-ray tracks.
Figure 7. Click to view. Cosmic-rays, through the extensive air showers they produce, are hypothesized to help increase the rate of cloud formation in the same process as a cloud chamber..

When cosmic rays come into the Earth's atmosphere, the muons ionize water molecules, thus changing their charge and causing them to be attracted to one another (see Figure 7). The water molecules condense into clouds. CERN is currently researching cosmic ray's relationship with cloud cover via project CLOUD: http://cloud.web.cern.ch/cloud/

An overview of DECO

While the cameras found in cellphones are designed to image in the visible portion of the spectrum, they can also detect other phenomena usually disregarded as noise for the purpose of taking pictures. The active element found in the cameras in cell phones is an array of charge-coupled devices (CCDs) which are sensitive to the visible portion of the electromagnetic spectrum. During the course of normal operation a CCD array will absorb photons and generate a small buildup of charge at each pixel corresponding to the intensity of the incoming photon.

Characterizing noise

However, CCDs will also generate a signal for incoming infrared photons as a portion of the infrared spectrum, as well high energy charged particles will also produce a signal which is generally disregarded as image noise on a digital camera. Part of what the software behind the DECO app does is to help characterize this background thermal noise, caused by ambient heat sources such as the phone's battery, in order to better establish a potential cosmic-ray signal.

With ambient visible light blocked, through any sort of opaque lens cover, and the background infrared noise taken into account one can begin to focus on the rarer but more intense signals associated with a potential cosmic-ray.

The software

Constantly, the Earth's surface is being bombarded with the remnants of extensive air showers. The primary component of which, once the shower reaches the Earth's surface, are muons. Muons are a heavy and unstable version of the electron which are often produced in the high energy collisions associated with the formation of extensive air showers. If a muon hits a CCD in a cellphone camera it, having both charge and energy, will be traveling with a great enough velocity that it will saturate every pixel it passes through leaving a white cluster or streak of pixels depending on the muon trajectory relative to the plane of the cellphone camera. The DECO app will then scan through successive images every few tenths of a second looking for a sudden burst of saturate pixels against a black background (The app doesn't generate any results unless the lens is covered with something such as thick, black tape).

The DECO app takes snapshots and rules out images in a non computationally-expensive way, then uploads possible candidates to our globalsensorweb.org servers. Aggregate analysis begins on data collected around the globe to determine, among other things, the origin and behavior of the radiation. In order for the app to run, the phone's screen must be covered and the phone should be charging. The easiest way to cover the screen is to merely place is face down on a surface, such as a night stand or table.

Figure 8 Figure 9
Figure 8. Click to view. Screenshot of the software running on the Android software.
Figure 9. Click to view. Detailed screenshot of the software running on the Android software.

Querying collected data

This was the method for extracting data. Currently we are moving over to a psql query system. To query data without headings here is a sample URL:
http://api.globalsensorweb.org:8080/v1/dataQuery.do?stationId=DECO-Android-1.0&startTime=2012/10/10 00:00:00 PST&format=text&options=with-headings

To query data with headings here is a sample URL:
http://api.globalsensorweb.org:8080/v1/dataQuery.do?stationId=DECO-Android-1.0&format=csv&options=with-headings

Please note:
1. Default startTime is now 00:00:00 of the current day.
2. Default endTime is now the current time.
3. Max interval is now 24-hours.

Testing the DECO application

Flux versus angle tests

During the summer of 2012 a series of tests were run on two Wildfire S phones, comparing the event rate versus the tilt angle of the phone. The phones were tilted from 0 to 60 degrees in 15 degree increments. While the event rate did decrease with tilt angle, as expected by comparing the event rate with purpose-built cosmic-ray detectors from Quarknet, the event rate for the Wildfire S phones did begin to rise slightly at approximately 45 degrees tilt relative to the ground. These preliminary results were presented by Veronic Hoyo at the Science Teachers And Research (STAR) conference at NASA-Ames in August 2012 (Conference poster linked here, raw data linked here)

Flux versus altitude tests

During the summer of 2013 ten more Android phones were added to the test population of phones for DECO. One method of testing the legitimacy of using DECO as a method for monitoring cosmic-rays is to compare the event rate on the phones with their altitude. This involves the placement of small groups of the phones at different altitudes and compare the variation with more established data as discussed through CERN and NOAA


Results of preliminary testing

During the holiday week of July 4th, two Samsung GT-I5500s were sent to Casper, Wyoming at an elevation of ~1,500m (~5,100ft). The phones ran for ~3 days. Unfortunately, only one event per phone was recorded. Upon returning to California, it was observed that one of the phones did not have an SD card, and therefore, the logged event was never saved. To see the query written in Postgres Standard Query Language (pSQL), click here. After attempting to query the event, it appeared as though the event never made its way into the database for unknown reasons.

Image Analysis - General

Each time the DECO app is run it takes an average pixel value from a thousand initial scans of the camera in order to calculate a 'floor' value. Currently, if a cluster of 1-100 pixels are detected above this floor then an event is detected. However, this threshold may be too low and has led to further tests as of the Summer of 2013.

In visually inspecting images by hand the color range is first limited in order to magnify the visibility of any noise and/or cosmic-ray track. Then a statistical check of the image is done in order to get a measurement of such quantities as the average pixel value, maximum pixel value, and the standard deviation on the pixel value. The first statistical check has been done with a Python script like this. In general, images which have readily identifiable cosmic-ray tracks have maximum pixel values well above the noise floor. Typically, if at least one pixel has a value above 150 (Out of the 0-255 color scale) then that image contains an identifiable track.

The raw images, with or without supposed tracks, look uniformly black until the color range is limited. When the color range is compressed it is far easier to differentiate image types and check for features such as tracks and noise due to heat generated by the phone. Thermal noise is easily identified as static, with a higher intensity closer to the edges of the image.

Figure 10 Figure 11
Figure 10. Click to view. Image with cosmic-ray track.
Figure 11. Click to view. Same image, but magnified and with the color range compressed.
Figure 12 Figure 13
Figure 12. Image with noise due to heating from the phone.
Figure 13. Click to view. Same image, but with the color range compressed.

Image Analysis - Current Processes, Motivations

PLACEHOLDER

Image Analysis - Thermal Flare

The CCDs in the cameras used in cellphones are sensitive to both visible light as well as part of the infrared band. This can cause a phone's camera, even if covered to block visible light, to register a signal on the CCD array from just the background heat of sources such as the phone's battery. The registered pixel values for thermal flare pixels are relatively low, typically below 20 out of 255 on any of the R, G, or B channels. They are therefore filtered out using the current pixel value cutoff of 150.

Image Analysis - Hot and Cold Pixels

The CCDs in a camera, being small electrical devices to convert an incoming light signal into a small electrical signal, can become damaged in a way in which they will not respond to incoming light signals. If this happens a pixel will typically register a consistent low value, as a cold pixel, or a consistent high value, as a hot pixel. In testing the efficacy of the DECO app it has been found that setting the event threshold value too low will cause hot pixels to be consistently detected as events. In studying the past two years of collected data, mid-2011 to mid-2013, there do not appear to be hot pixels with values of greater than 150 out of 255. Setting the RGB filters at or above that level appears to fix this issues. Cold pixels, while removing potential data from the camera, will not yield false positives.

Image Analysis - Generating Histograms

In order to properly discriminate between cosmic ray events and false positives both a human and machine readable way of observing the distribution of data values in order to establish accurate thresholds and parameters for data acquisition and analysis. Therefore a script was developed to generate histograms, or visual representations of the data's distribution, for human consumption and properly formatted CSV files for machine consumption. Histograms are created by taking an image or set of images and running it through the Image Histogram Generator script detailed below.

Image Analysis - "Worms"

Electron worms are curved tracks that result from the terrestrial beta decay of radioactive material. As they are not cosmic in origin they are of low enough energies to have their activity interacting with molecular structures observable to the CCD. To read more, see this study by Lawrence Berkeley National Laboratory and UC Santa Cruz.

Figure 13 Figure 14
Figure 13. Click to view. Example of electron worm.
Figure 14. Click to view. Example of electron worm.

Gallery of interesting images

|
Figure 12. Image with noise due to heating from the phone.


Other resources

Scripts

These are the python scripts currently being used for simple image analysis and event filtering

Image Statistics script

This script logs

  • Image size, Image width, Image height,
  • Max subpixel value,
  • Position of max subpixel
  • Minimum subpixel value and position of that subpixel
  • Average value of pixels
  • Relative variation in pixels
  • Standard deviation of pixels

to a csv file to allow easy access to certain statistical values for easy access.

import pylab
import numpy
import os
import glob
import csv

#gets current path
path = os.getcwd()
#opens csv file and write a header
c = csv.writer(open("DECOImageAnalysis.csv", "wb"))
c.writerow(["Image size","Image width","Image height","Max image value","Position of max image value","Min image value","Position of min image value","Average value","Relative variation","Standard deviation"])
    #The for loop finds all the files ending in .jpg and writes statistical values of that image on the csv
for infile in glob.glob( os.path.join(path,'*.jpg') ):
    print "current file is: " + infile
    img=pylab.imread(infile)
    relVar = img.mean()/img.std()
    c.writerow(["%6d"%(img.size),"%3d"%img.shape[0],"%3d"%img.shape[1],"%1.2f"%img.max(),"%6d"%img.argmax(),"%1.2f"%img.min(),"%6d"%img.argmin(),"%1.5f"%img.mean(),"%1.5f"%relVar,"%1.5f"%img.std()])

Image filtering by RGB value

This is another python script that looks at all the images in the current directory. It writes some of the same statistical values as the Image Statistics script except it also logs the Pythagorean sum of the three brightest subpixels (which we call Color distance). However, it only does this is the brightest subpixel is greater than a certain cutoff, currently 150 out of 255. This hopefully allows basic filtering of events vs nonevents, however electron worms are not able to be filtered out currently. The value of 150 hopefully will get rid of images that are just stuck pixels.

import pylab
import numpy
import os
import glob
import csv
import time
import math
import Image
 
#Get current path
path = os.getcwd()
#open csv and write header
c = csv.writer(open("DECOImageAnalysis.csv", "wb"))
c.writerow(["Image number", "Event Time","Image size","Image width","Image height","Max image value","Position of max image value","Min image value","Position of min image value","Average value","Relative variation","Standard deviation","Red Max","Green Max","Blue Max","Color distance"])
#finds each file ending in .jpg and opens them
for infile in glob.glob( os.path.join(path,'*.jpg') ):
    print "current file is: " + infile
    im = Image.open(infile)
    #get 2d array with the subpixel values of the brightest pixel
    maxColor = im.getextrema()
    redMax = maxColor[0][1]
    greenMax = maxColor[1][1]
    blueMax = maxColor[2][1]
    #calculate the Pythagorean sum of the brightest subpixels and writes that as well as image 
    #statistics to a csv as long as at least one subpixel is greater than 150
    if (redMax > 150 or greenMax > 150 or blueMax > 150):
        colorDistance = math.sqrt(redMax**2 + greenMax**2 + blueMax**2)
        print colorDistance
        img=pylab.imread(infile)
        relVar = img.mean()/img.std()
        f = open(infile)
        data = f.read()
        creationTime = time.ctime(os.path.getmtime(infile))
        f.close()
        c.writerow([os.path.splitext(infile)[0], "%s"%creationTime,"%6d"%(img.size),"%3d"%img.shape[0],"%3d"%img.shape[1],"%1.2f"%img.max(),"%6d"%img.argmax(),"%1.2f"%img.min(),"%6d"%img.argmin(),"%1.5f"%img.mean(),"%1.5f"%relVar,"%1.5f"%img.std(),"%1.5f"%redMax,"%1.5f"%greenMax,"%1.5f"%blueMax,"%1.5f"%colorDistance])

Image Histogram generator

This is a script that looks at each file in the current directory and generates a histogram. The results are graphed as well as saved to a csv file. This gives a was to visually see statistical differences between events and nonevents. In the graph, the x axis represents pixel brightness, and the y axis shows the number of pixels with the corresponding brightness. The first 255 values are for the red pixels. The next 255 are for green, and the last 255 are for blue. This means that point 256 on the x axis is actually 0 for the green subpixels, and point 512 is 0 for blue. This script uses the histogram() function from the Python imaging lybrary, and matplotlib to plot the graph. These libraries are required to run the script.

from PIL import Image
import pylab
import numpy
import os
import glob
import csv
import os.path
import time
import math
import Image
import matplotlib.pyplot as plt

#Get current path
path = os.getcwd()
#open csv
resultFile = open("DECOImageAnalysisHistogram.csv",'wb')
wr = csv.writer(resultFile, dialect='excel')

#Goes through each jpeg in the current directory and writes a histogram to the csv 
#using the .histogram function of the Python Imaging Library.  
#Then a graph is generated and saved
for infile in glob.glob( os.path.join(path,'*.jpg') ):
	print "current file is: " + infile
	im = Image.open(infile)
	pixels = im.load() 
	hist = im.histogram()
	wr.writerow(hist)
	
	#create a new pyplot figure and subplot to store the graph
	fig = plt.figure()
	imgHyst = fig.add_subplot(111)
	
	#get the length * width of the image's pixels 
	#to get the Y axis maximum for the graph
	size = im.size
	sizeXY = size[0] * size[1]
	
	#Plot the hist array.  The first 255 values are red subpixels, 
	#the second 255 are green and the last 255 are red.
	#They are plotted in separate colors
	imgHyst.plot(numpy.arange(0, 256), hist[0:256], 'ro', label="red subpixels")
	imgHyst.plot(numpy.arange(256, 512), hist[256:512], 'g^', label="green subpixels")
	imgHyst.plot(numpy.arange(512, 768), hist[512:768], 'bs', label="blue subpixels")
	
	#set the axis scale and size
	imgHyst.axis([0, 767, 0, sizeXY ])
	imgHyst.set_yscale('log')

	#Set ticks on x axis to a frequency of 64
	imgHyst.set_xticks([0, 64, 128, 192, 256, 320, 384, 448, 512, 576, 640, 704, 767])
	
	#add labels and save as png
	imgHyst.set_xlabel("Pixel value")
	imgHyst.set_ylabel("Number of pixels with value x")
	imgHyst.set_title('RGB Histogram')
	imgHyst.legend()
	fig.savefig(infile + "hist.png", format='png')
                   

Talks given on DECO

A recent talk given on an overview of the DECO project, as a citizen science project, given by Zach Irwin on the 10th of March 2013 is posted here. A talk given to the American Geophysical Union conference in San Francisco on 13 December 2013. Presentation available here.

Established detection systems

Notes

Template:Reflist