AGU Fall Meeting 2011 and around that time

View from the airport (SFO)

Line to buy BART tickets

My hotel room

Moscone West hall

Poster Hall

My poster

Alkatraz

Golden Gate bridge

find

Linux: search for a folder with name afb:

To which cell the point belongs

Recently I was relating measurement points and grid cells, using shapely's Polygon.intersect method. And was lucky enough to bump into the situation where the point, which corresponds to a real observation station, was not inside any grid cell. Though it was inside the simulation domain judging by its coordinates. Fortunately we got Polygon.buffer method, which allowed me to extend the polygons, in order to make sure that every point inside the domain will fall into some cell. The length of the buffer was selected as 0.01 * (cell edge).

How I implemented 2d affine transform in python

Although there is nothing special about it, but, strangely, I could not find it in GDAL-python (though it is very good library for working with raster and shape files, and I like it), also I looked at another respected library - shapely, but unfortunately, there is no capability to move, rotate or translate the shapes (but still there are some very handy functions like union, intersection, distance, area, ..). Another thing is that this can be easily done in Java using JTS, like this:
first I construct a polygon at the origin of the coordinate system and then determine the transformation needed to obtain this unit rectangle from the initial polygon (It can be defined uniquely using height of the polygon and the coordinates of the centers of the parallel edges). Then using the inverse transform we obtain the geometry object corresponding to the specification above (height, coordinates of the centers of the parallel sides). I drew these polygons using their wkt representation in OpenJump.

And still I decided to use python, since there exists an easy way of displaying results in matplotlib. Some people use descartes library for this purpose, but I don't feel like installing another dependency, since the thing with python is that if you change the OS, you have to reinstall them all. For me it is: matplotlib, numpy, scipy, gdal, cdat, shapely, netcdf4-python.... Well, I am getting distracted from the main point. So briefly, I feed my geometries to matplotlib as follows:

So the transformation I did, can be initialized using 3 source and 3 corresponding destination points, that shoud not lie on the same line. Tested it, kind of works. Probably will optimize and improve it later, if necessary, but for now it looks like this:
Basically, I just solve a system of linear equations with respect to parameters of the transformation, and then use the parameters to transform other points.

Adding text files

If you have many files with text information, it is easier to look for what you need, if
all the info is assembled into a single file. To achieve this in Bash, you can do the following:

this will put all files from the current directory into one file.
More complicated manipulations, such as sorting and elimination of duplicates,
described here.

Bash: kill processes with a given name pattern

As we know the kill command in linux permits to kill a process using its id. kill -9 tries to kill your process even harder. But what if I have many similar processes? We can use the cut function that I recently saw here as shown below:

What it does is: searches the list of processes for lines containing bootstrap.py, then extracts the process id from each line and kills the process.

C++: Allocating 2D array using new

This link explains how to allocate 2d arrays with variable dimensions:
the link. I find that it is a very useful and clear explanation.

NCO

NCO is a very powerfull tool for editing netcdf files. In this post I will be writing some useful commands.

1. Delete a dimension of size 1 (the dimension name is level):

ncwa -a level directions_qc_amno.nc directions_qc_amno1.nc

2. Copy data from one to the other netcdf file, if possible recognize common dimensions:

ncks file1.nc file2.nc

3. Rename a dimension

ncrename -d rlat,latitude directions_qc_amno.nc

4. permute dimensions(make longitude first and the latitude second):

ncpdq -a longitude,latitude infocell.nc infocell.nc 

5. Rename the variable lat to latitude:

ncrename -v lat,latitude infocell.nc

6. Take difference of variable x in 2 files:

ncbo -v x --op_typ='-' infocell.nc infocell_1.nc infocell_diff.nc

7. Operation on variables in a file. Here the values of the variable are changed and the attributes added.

ncap -O -s "air=(air-512.81)/0.01" filename.nc temp.nc
ncap -O -s "air=short(air);air@add_offset=512.81;air@scale_factor=0.01" temp.nc filename.nc

taken from http://jisao.washington.edu/data/nco/
8. Concantenating netcdf files along the record dimension.

ncrcat runoff_file_name_pattern total_runoff_af.nc

9. Subsetting by index and by coordinate values:

ncks -d lon,0,4 -d lat,2,9,2 in.nc out.nc
ncks -d lon,10.,20. -d lat,80.,90. in.nc out.nc

Used data from:

• http://linux.die.net/man/1/ncbo
• http://nco.sourceforge.net/#RTFM