1. List content:

> ls -lt *.gs (to list all GrADS scripts in current directory and sort them by time)
> ls -lt | head (to list the files modified most recently)
> ls -lt *.gs | head -20 (to list the .gs files modified most recently) 
> ls -lt | grep ldassa (to list the files containing "ldassa" in the filename?)
> ls -1t *.txt (number 1 means to just give you the names, not including other info e.g. size and directory)
> ls -l | grep pft | grep ease (to list the files/directories? that have pft and ease in the name)
> ls -lt | more (then use the space key to go to next page) 

2. Search:
 
> grep cn0045 * (to search for files that have 'cn0045' under current directory)
> grep -r forecast (to search for files that have 'forecast' under current directory and subdirectories)
> /bin/grep CN_LAI *.f90 | grep dat (to be case sensitive)
> find . -iname catchment.F90 (to search for a file called catchment.F90)
> find . -type f | wc (to count the number of files, wc means word count)
> find . -type f -exec grep -i GEOS_CatchGridComp {}\; (to search for which files call GEOS_CatchGridComp)
> find . -type f -print -exec grep -i GEOS_CatchGridComp {}\; (to search for which files call GEOS_CatchGridComp, this print out all the filenames)
> find . -type l (to search for all the symbolic links in the current directory)
> find . -type l -exec /bin/rm {}\; (to search for and remove all the symbolic links in the current directory. NOTE: DO NOT do this unless you are sure that you are removing the ones that you don't want. Use find . -type l to take a look at what you are going to remove first)
> find . -iname '*.F90' -exec cat {} \; >> code.all (to put the content in all .F90 files in one file code.all)
> find . -type f -exec cat {} \; >> ../model.all (put the content of all files in one file model.all. NOTE: do this before you compile!)
> find .type d -exec chmod 755 {} \; (to locate all directories (-type d) under the current directory (.) and run chmod 755 on them)
> find /discover/nobackup/fzeng/geos5_code/Icarus-1_0_p1/ -name "*.pl"  (to find all the Perl files under /discover/nobackup/fzeng/geos5_code/Icarus-1_0_p1 and the subdirectories)

3. Remove:

> /bin/rm -r directory (to remove a full directory)
> rmdir directory (to remove an empty directory)
> /bin/rm -rf RUN e0004_*.gdat (to remove a full directory and some files at once)
> rm /--append (to remove a file with name starting with symbols, do this in the directory where the file is located)

4. Check size:

> du -sk geos5 (to check the size of a directory)
> du -sk * | sort -n (to check the size of files and directories in current directory and sort them by size)

5. Compress:

> tar cvf ../temp.tar e0005s e0006s e0007s e0008s (to tar the four directories to temp.tar)
> tar ztvf  tile_dail.201504.tgz (to list what is in this .tgz file in $ARCHIVE)
> tar xf *.tar (extract the *.tar to current directory)
> tar -xvzf /path/to/yourfile.tgz (to extract the .tgz to current directory)

6. Copy:
/discover/nobackup/gkwalker/cvs > cp -pr b_gkw_Heracles-4_0-CN /discover/nobackup/fzeng/. (to copy the entire directory)
scp -p fzeng@land01.gsfc.nasa.gov:/terra/fzeng/filename .  (to copy from land01 to discover)

7. For debug purpose:

in process-smap (any shell script), add 'x' after '#!/bin/csh' (i.e. the first line, now it reads '#!/bin/csh x'), it will print out all the steps on screen when process-smap is running.

8. Look at difference:

> diff -arq (to take a look at the difference in two directories)
> diff -q file1 file2 (to check if the two files are identical)
> diff --brief -r dir1/ dir2/ (to get a list of corresponding files whose content differs)
> diff --brief -Nr dir1/ dir2/ (to get a list of corresponding files whose content differs and see differences for files that may not exist in either directory)
> cmp ?.x ??.x (to compare two executables)
> ~smahanam/bin/cdo -diffn netcdf1 netcdf2 (to compare two files field by field)

9. netCDF:

(1) Show netCDF file content:
see notes on 20150819

How to tell whether a catch_internal_rst file is netcdf or binary (see notes on 20150709):
Run the commands below ((L387-388 in gcm_run.j in h4-0000)) on any of the restarts (e.g. catch_internal_rst) in the directory where the restart files are located, it will tell you if it's .hdf , binary or netcdf: 

set      fvrst = `cat AGCM.rc | grep "^DYN_INTERNAL_RESTART_FILE"  | cut -d ":" -f2` (L387 in gcm_run.j in h4-0000)
set filetype = `/usr/bin/file -Lb --mime-type $fvrst`
echo $filetype

If it returns "appliation/x-hdf", it means that the file type is netCDF (hdf?).
If it returns "application/octet-stream", it means that the file type is binary. 

(2) Check the version:
ncdump -k *.nc

(3) Show the data of a variable:
ncdump -v varname filename

On land01:
ncview **.nc & (to view the data)

10. Check server usage:

> top (to check the top users on land01)
land01:~> df (to check the usage of land01, including /home, /land, /terra and others)

11. Check password expiration date and change password on land01, land02 and train:

> chage -l fzeng (to check expiration date)
land01:~> passwd (to change password)

> passwd (to change password on NCCS)

12. Linux Rename Multiple Files At a Shell Prompt:

/discover/nobackup/fzeng/Catchment/merra2/m0008s_01 > rename m0008 m0005 *.gdat

13. To move running process to background:
Ctrl+z
> bg

14. To view image files on discover:
Can use "display" command;
For pdf files, use "evince" command for "/usr/local/other/SLES11/xpdf/3.03-gcc-4.3.2/bin/xpdf".
For *.ps files created by IDL, use "gv *.ps".

15. Archive:
> dmget file(s)_in_archive (to retrieve files from archive to the current directory)
> dmls -l file (to check if the file is on disk or on tape)

?. Miscellaneous:

> head filename (to show the top part of the file)
> cmp file1 file2 (to compare two files of any type)
> df -h /terra (to check the usage of terra)
> ~/merra2 > groups (to see what groups I am in)
zz or :wq (to quit vi and save the contents of the buffer to the file that vi is being used to edit)
> touch -d '1 May 2015' filename (to set the time stamp of filename to 1 May 2015)
(see more in http://www.linfo.org/touch.html)
> finger $lastname (to take a look at the path of somebody's home directory)
> cd - (to go back to the last directory visited)
> nedit e000?s/setup & (to open the files named "setup" in all the e000?s directories)
> qa | grep gkwalker (for me to take a look at Greg's jobs)
> module list (to show what modules are loaded)
> echo $shell (to take a look at what shell I use)
> ulimit (to take a look at the limit of the stack size?)
> whick ifort (to see what version of ifort I use)
> env (to take a look at my run environment)
> ~bmauer/bin/cdo infon **.nc4 (to display the statistics, e.g. min, mean and max, of the variables in a nedCDF file)
> ~atrayano/bin/tell_rec catch_internal_rst (to find out number of tiles)
> csh (to enter c shell, exit or logout to exit c shell)
> ncdump -h *.nc4 (to take a look at the information about a netCDF file and its content)
> ncdump -v ARS1 catch_internal_rst > tmp.txt (export the values of ARS1 in catch_internal_rst to tmp.txt for display)
> ~bmauer/bin/ack leaf_mr (search for the routines that have "leaf_mr" in the current directory, better than grep)
> lncdump -c *.nc (what does this do?)
> cmp *.x *.x (to compare two executables)
> xterm & (to open a new terminal window that I already signed in, so no need to enter username or password again)
> module av tool (to check what modules are available under directory tool/)
> module load tool/matlab-R2011b 
> lscpu (to take a look at the number of cpus on a dali node)
> xeyes (to check if it can display graphics)
> fg (to resume the most recently suspended or backgrounded job)
> ps -ef | grep tot (to find the job number of totalview)
> ncdump -v varname ncfilename (to view the values of a variable in a netcdf file)
> tail -f filename (to display the last part of the file, content displayed will change as the file is being updated)
> qselect -u fzeng | xargs qdel (to delete all my jobs at once)

> dmls -l (to see if the directories are DUL - on disk, or OFL - on tape. Slurm can see anything on tape)
> dmget (works as copy plus untar)

To clean the memory:
AGCM:
> ~mathomp4/bin/larrykillall
GEOSldas:
> ~smahanam/bin/ldaskillall

command + k (to clear the terminal)

Ctrl+A: get to the beginning of a terminal command line
Ctrl+E: get to the end
Ctrl+F: move the cursor forward word by word
Ctrl+B: move the cursor backward word by word

To load a particular matlab version:
land01:/ford1/local/matlab_r2014a/bin> ./matlab & (to load matlab r2014a on land01)
or /ford1/local/matlab_r2014a/bin/./matlab & (from anywhere on land01, e.g. 
land01:/land/fzeng/projects/fpar/code>/ford1/local/matlab_r2014a/bin/./matlab &)

Check the status of my runs today:
> sacct

Check which group I belong to:
> groups

Check the project that sponsors me:
> getsponsor

Check allocation:
> allocation_check

Check if it's sp1 or sp3: uname -a
dali16:/discover/nobackup/fzeng > uname -a
Linux dali16 2.6.32.54-0.3-default #1 SMP 2012-01-27 17:38:56 +0100 x86_64 x86_64 x86_64 GNU/Linux
dali16:/discover/nobackup/fzeng > ssh discover-sp3
discover07:~ > uname -a
Linux discover07 3.0.101-0.40-default #1 SMP Thu Sep 18 13:09:38 UTC 2014 (44b8c95) x86_64 x86_64 x86_64 GNU/Linux
The discover07 is "3.0**", so it's sp3.
sp: service pack
If you compile the code on sp3, you need to run on sp3.
Haswell is sp3, so we have "constraint=Haswel" in the job scripts. 

Discover system details:
http://www.nccs.nasa.gov/discover_front.html

To kill a job running in the background: 
xterm & (to open a new terminal window on the same node)
ps -u (to show a list of jobs under my account)
find the job number for "mpirun ..." and then kill it using "kill jobnumber".

To kill a job running interactively:
> Jobs (to list all the jobs)
> Kill %2 or fg %2 (to kill job 2)

To kill a job running in background on land01:
> ps -ef | grep fzeng (to bring up a list of the jobs I currently have)
> kill -9 jobnumber (the jobnumber is the number in the 2nd column; the 1st column is fzeng) 
> ps -ef | grep fzeng again to double check it's killed. 

To check my quota on archive:
~/Catchment > ssh dirac
/home/fzeng> quota

How to calculate the size of a data file:
If the dimension of the data is 144 by 73, and there are 168 months (14 years), and only one variable, the size of the data file should be close to 144x73x4x168 = 7,064,064 (4 is 4 bytes per record). If there are more than one variables, this should be further multiplied by the number of variables. 


There is matlab, and other softwares, on land01:/ford1/local.


References:


GEOS-5 wikipage:
geos5.org/wiki

http://geos5.org/wiki/index.php?title=Ganymed_4.1_Quick_Start

https://progress.nccs.nasa.gov/keyupload/