========== 20180103: 1. Met with Eunjee to talk about the plots I made to compare the offline Catchment-CN results with FLUXCOM GPP and GIMMS FPAR for the 2011 and 2012 US drought events. 2. December report. 3. Updated Higgins rain data throught 2 Jan 2018. 4. Continued investigating why the GPP in the upgraded Catchment-CN is so low. See /discover/nobackup/fzeng/clm4-to-clm4.5/notes/notes_daily. ========== 20180104: 1. Met with Randy and Eunjee about the drought paper. 2. Met with Randy about the low GPP in CLM4.5. (1) About FPAR, parsun and parsha: Yes. The parsun and parsha that go into subroutine Photosynthesis are already per vai (vai = elai + esai). And after calling compute_rc, psn (photosynthesis) is computed as psnsun(:,nv,nz)*laisun(:,nv,nz) + psnsha(:,nv,nz)*laisha(:,nv,nz) Therefore, actually only the PAR absorbed by leaves is used to compute GPP. This makes sense and is consistent with CLM4. (2) For Jan 1980, CLM4.5 GPP / CLM4 GPP = 0.7568 (used ~/Catchment/CLM4.5/debug/compute_diff.gs interactively). If this ratio holds for the entire 1980-2016 period and if CLM4 GPP is ~132 PgC/yr, CLM4.5 GPP will be 132 * 0.7568 = 100 PgC/yr. Original CLM4 and CLM4.5: 165 PgC/yr * 0.7568 = 125 PgC/yr, close to 130 PgC/yr that Bonan et al., 2011 reported. 3. Processed Jan 2018 seasonal forecast. 4. GMAO holiday party. ========== 20180105: 1. Helped Eunjee set up and automate the transient CO2 run. Checked my notes for 201510 and 201511 but could not find the details on how I automated transient CO2 run before. Recovered m0009re_33_p01 for an example. cd /discover/nobackup/fzeng/Catchment/merra cp -p /archive/u/fzeng/Catchment/merra/m0009re_33_p01.tgz . tar -xf m0009re_33_p01.tgz rm m0009re_33_p01.tgz Email to Eunjee on 9/26/2016: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (1) The executable is /discover/nobackup/fzeng/offline_code/LDASsa_m3-15_2-CN/exec/LDASsa_mpi_M2n5P_transientCO2.x. (2) The CN and land restart files are from your 280ppm spinup. (3) Please refer to /discover/nobackup/fzeng/Catchment/M2n5P/m0001_transientCO2_neUS/lenkf.j for an example of the job script. In this first cycle you will use meteorology from 1991-2015 (set stopyear to 2016 in the job script). Could you cross check if the CO2 years and Met years in the table below are correct? I save 3-hourly output for testing purpose, but you don't need that. You may just save monthly output. An example of the namelist for saving monthly tile output is ~/Catchment/LDASsa_m3-15/M2n5P/LDASsa_inputs_driver-mon.nml. Please make sure that "model_dtstep = 600" in L31. You may be able to finish 2 years of simulation within 12 hours, hopefully. (4) Do "echo 1851 > year_co2.txt" under your experiment directory to create year_co2.txt which has 1851. Then you can submit your job. Let me know if you have any questions. I will show you an example of the post-processing script which converts tile-space output to grid-space later. Since there will be only 3 full (i.e. 34 years from 1981-2015) cycles, I think it's better to set up the next cycle manually every time a cycle is done. However, if you would like to set up automatic submit using submit_next_batch_**, I can show you how to do that. In that case you will need to carefully monitor when it running CO2 years 1946-1980. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Recovered m0001_transientCO2_neUS from archive: cd /discover/nobackup/fzeng/Catchment/M2n5P/m0001 cp -p /archive/u/fzeng/Catchment/M2n5P/m0001/m0001_transientCO2_neUS.tgz . tar -xf m0001_transientCO2_neUS.tgz rm m0001_transientCO2_neUS.tgz Also referred to my notes on 20160926. Test it myself first: cd /discover/nobackup/fzeng/Catchment/M2n5P/m0001 mkdir -p m0001_transientCO2/RUN/rs/ens0000/Y1991/M01 cd m0001_transientCO2 cp -p /discover/nobackup/elee15/offline/sims/co2var_spinup/M2.n5P.HRv2.280ppm.c14/CN_restart . cp -p /discover/nobackup/elee15/offline/sims/co2var_spinup/M2.n5P.HRv2.280ppm.c14/RUN/rs/ens0000/Y2016/M01/mM2.n5P.HRv2.280ppm.ens0000.catch_ldas_rst.20160101_0000z.bin RUN/rs/ens0000/Y1991/M01/m0001.ens0000.catch_ldas_rst.19910101_0000z.bin Checked the restart files: dali15:/discover/nobackup/fzeng/Catchment/M2n5P/m0001/m0001_transientCO2 > ls -l CN_restart -rw-r--r-- 1 fzeng g0620 1530820456 2018-01-04 16:06 CN_restart dali15:/discover/nobackup/fzeng/Catchment/M2n5P/m0001/m0001_transientCO2 > ls -l RUN/rs/ens0000/Y1991/M01 -rw-r--r-- 1 fzeng g0620 41453776 2018-01-04 16:06 m0001.ens0000.catch_ldas_rst.19910101_0000z.bin echo 1851 > year_co2.txt cat year_co2.txt (to make sure the year of CO2 is correct) cp ../m0001_transientCO2_neUS/lenkf.j . nedit lenkf.j & Email Eunjee the procedures and tell her to refer to my /discover/nobackup/fzeng/Catchment/M2n5P/m0001/m0001_transientCO2 and /home/fzeng/Catchment/submit_next_batch_M2n5P_transientCO2. NOTE: Don't use submit_next_batch_M2n5P_transientCO2 in the first cycle. Need to mannually set up the 2nd cycle and change the STRTYEAR to 1981. For the last cycle, not sure if this line "if(${NEXTYEAR_co2} >= 2001) exit 14" can stop the run at the end of 2000 correctly (haven't tried). ========== 20180108: 1. Re-made all the anomaly maps. Now for the 2011 Texas drought the climatology is 2001-2011 for all the fields, and 2001-2012 for the 2012 central-eastern US drought. Also included 2 more months to have a better idea on how long the drought lasted, so 7 months total now. 2. Continued CLM4.5 work. See /discover/nobackup/fzeng/clm4-to-clm4./notes/notes_daily. 3. Read Sarith's process_seasonal_forecast.csh and s2s_fcast.pro in /discover/nobackup/fzeng/GEOS5_s2s/src to understand hindcast data. Didn't quite understand. Will read Greg's scripts in ~/geos5/hindcasts to learn hindcast data. $HOME/geos5/hindcasts/extract_p_t and $HOME/geos5/hindcasts/extract_p_t.gs ========== 20180109: 1. Monitoring the 35th cycle of clm4.5_DE720. It is simulating 20090928 at ~1pm. Will make a minor change to FPAR calculation (FPAR = FPAR * elai/(elai+esai)) after it finishes. 2. Double checked compute_rc.F90 (including subroutine Photosynthesis but excluding the subroutines hybrid etc.) by comparing to the original CLM4.5 CanopyFluxesMod.F90, SurfaceRadiationMod.F90 and SurfaceAlbedoMod.F90. Didn't see any bug. Re-visited the Bonan et al., 2011 paper. The forcing data used in Bonan et al., 2011 is the dataset described by Qian et al., 2006. ========== 20180110: 1. clm4.5_DE720 finished the 35th cycle. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5 mv clm4.5_DE720_21258375 clm4.5_DE720_35 cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720/input/restart rm output ln -s /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720_35/output Compute 1980-2016 mean global GPP and NEE of clm4.5_DE720_35 using read_fields.m and calculate_global.m in ~/Catchment/matlab/clm4-to-clm4.5. 2. Modified compute_rc.F90: (1) changed "parabs(n) = parabs(n) + (pardir(n)*fabd + pardif(n)*fabi)*fveg(n,nv)" to "parabs(n) = parabs(n) + (pardir(n)*fabd + pardif(n)*fabi)*fveg(n,nv)*wl". (2) removed all the "tile_id" (3) updated the equation numbers to match CLM4.5 Tech Note Modified process_cn.F90: (1) Remove "tile_id" from the line calling compute_rc Compiled the code: "gmake clean" and "gmake install" /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4.5/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 > /bin/cp -pr Linux/ exec/clm4.5/. 3. The 36th cycle was already set up by the submit_next_batch_DE720 script when the 35th cycle finished. Double checked the executable and restart file and submitted it. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720 ls -l build/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 71777037 2018-01-10 11:59 build/Linux/bin/LDASsaCN_mpi.x* Good. ls -l input/restart/ lrwxrwxrwx 1 fzeng g0620 80 2018-01-10 09:55 output -> /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720_35/output/ Good. cd run qsub lenkf.0.j 4. Read Qian et al., 2006, Sheffield et al., 2006, CLM4 and CLM4.5 Tech Notes. Global GPP in PgC/yr: Bonan et al., 2011 Catchment-CN -------------------------------------------------- Forcing Qian et al. 2006 MERRA2 with SMAP N5 precip CLM4 165 132.5 CLM4.5 130 ~100 CLM4.5/CLM4 0.788 0.757 (based on Jan 1980 GPP) GPP is higher in Bonan2011 for both CLM4 and CLM4.5 compared to Catchment-CN forced by MERRA2 reanalysis. This is likely due to this assumption "the ratio of visible to total incident solar radiation is assumed to be 0.5". 5. Helped Eunjee set up the 2nd cycle of the transient CO2 run. 6. Understanding hindcasts meteorology: Read Greg's scripts in ~/geos5/hindcasts to learn hindcast data. $HOME/geos5/hindcasts/extract_p_t and $HOME/geos5/hindcasts/extract_p_t.gs ========== 20180111: 1. Checked the FPAR in the current (36th) cycle of clm4.5_DE720 by comparing it to FPAR in the 35th cycle. There is quite a bit of reduction in FPAR. This is good. The high biases in CLM4.5 FPAR will be largely reduced, and now CLM4.5 FPAR is consistent and comparable with CLM4 FPAR. 2. Read MERRA-2 File Specification file and clsm_ensdrv_force_routines.F90 to understand what exactly we need to drive Catchment-CN and whether Hindcasts has everything we need. Asked Lesley and Lena for help. Lena: The new S2S v2.1 system does NOT produce DRPAR and DFPAR (direct and diffuse PAR). Resolution is 361*720. The old Fortuna system produced DRPAR and DFPAR, but at a different resolution, 181*288. ========== 20180112: 1. Continued collecting information about the Hindcasts data. 2. Emailed Justin to check if what's said in the CLM4 and CLM4.5 Tech Note files that "the ratio of visible to total incident solar radiation is assumed to be 0.5" is true in the original CLM4 and CLM4.5 source code. Justin confirmed this and pointed me to the original CLM code, /models/atm/datm/datm_comp_mod.F90: !--- shortwave radiation --- if (sswdndf > 0 .and. sswdndr > 0) then a2x%rAttr(kswndr,n) = avstrm%rAttr(sswdndr,n) * 0.50_R8 a2x%rAttr(kswvdr,n) = avstrm%rAttr(sswdndr,n) * 0.50_R8 a2x%rAttr(kswndf,n) = avstrm%rAttr(sswdndf,n) * 0.50_R8 a2x%rAttr(kswvdf,n) = avstrm%rAttr(sswdndf,n) * 0.50_R8 elseif (sswdn > 0) then ! relationship between incoming NIR or VIS radiation and ratio of ! direct to diffuse radiation calculated based on one year's worth of ! hourly CAM output from CAM version cam3_5_55 swndr = avstrm%rAttr(sswdn,n) * 0.50_R8 ratio_rvrf = min(0.99_R8,max(0.29548_R8 + 0.00504_R8*swndr & -1.4957e-05_R8*swndr**2 + 1.4881e-08_R8*swndr**3,0.01_R8)) a2x%rAttr(kswndr,n) = ratio_rvrf*swndr swndf = avstrm%rAttr(sswdn,n) * 0.50_R8 a2x%rAttr(kswndf,n) = (1._R8 - ratio_rvrf)*swndf swvdr = avstrm%rAttr(sswdn,n) * 0.50_R8 ratio_rvrf = min(0.99_R8,max(0.17639_R8 + 0.00380_R8*swvdr & -9.0039e-06_R8*swvdr**2 + 8.1351e-09_R8*swvdr**3,0.01_R8)) a2x%rAttr(kswvdr,n) = ratio_rvrf*swvdr swvdf = avstrm%rAttr(sswdn,n) * 0.50_R8 a2x%rAttr(kswvdf,n) = (1._R8 - ratio_rvrf)*swvdf else call shr_sys_abort(subName//'ERROR: cannot compute short-wave down') else call shr_sys_abort(subName//'ERROR: cannot compute short-wave down') endif Note swndr is near-IR direct, swvdr is Visible direct, likewise swndf and swvdf are similar for diffuse radiation. 3. Global GPP and NEE of the SMAP M09 simulations: transientCO2_05 ~/Catchment/matlab/SMAP/read_fields.m Recorded and plot the numbers in the excel sheet on desktop. ========== 20180116: 1. Checked the GPP and NEE in the 36th and 37th cycles of clm4.5_DE720. 2. Updated higgins rain data through Jan 15, 2018. 3. Used ~/Catchment/matlab/SMAP/read_fields.m to compute global annual GPP and NEE for the SMAP M09 simulations: preindustrialCO2_19 transientCO2_03 transientCO2_04 Recorded and plot the numbers in the excel sheet on desktop. 4. Investigated why burned area is so high in CLM4. See notes in /discover/nobackup/fzeng/clm4-to-clm4.5/notes/notes_daily. ========== 20180117: 1. Used ~/Catchment/matlab/SMAP/read_fields.m to compute global annual GPP and NEE for the SMAP M09 simulations: transientCO2_01 transientCO2_02 2. IDS meeting 3. Met with Randy. 4. Examined the impact of the assumption that half of SWdown is IPAR on GPP. So the clm4_IPAR_DE720 experiment below uses MERRA-2 with SMAP N5 precipitation, and scales PARdir and PARdif using the assumption of IPAR(i.e. vis)/SWdown=0.5. Modified process_cn.F90 in /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3. Compiled. /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 > /bin/cp -pr Linux/ exec/clm4_IPAR/. Set up the experiment: cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run nedit D0.5.bat D0.5_cn.exe & cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/exec/clm4_IPAR/Linux/bin ./ldsetup setup /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5_cn.exe /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5.bat --runmodel --monthsperjob 96 --landmodel catchCN cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/input/restart ln -s /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_DE720_19/output [use the last restart file of clm4_DE720_19] rm -f rst_clm4 cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/run nedit lenkf.0.j & changed "-restart_path ../input/restart/rst_clm4 \" to "-restart_path ../input/restart/output \" change "-restart_id clm4_IPAR_DE720 \" to "-restart_id clm4_DE720 \" [forgot this and the consequence can be seen in the 20180119 notes] cp -p ../../clm4_DE720_19/run/edit_lenkf.csh . cp -p ../../clm4_DE720_19/run/LSM_Calib.rc . cd .. mkdir lenkf_orig cd run cp -p lenkf.*.j ../lenkf_orig/. ./edit_lenkf.csh remove the last line "/home/fzeng/Catchment/CLM4/submit_next_batch_DE720 clm4_DE720" from lenkf.4.j Check the executable: ls -l /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/build/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 91790816 2018-01-17 17:06 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/build/Linux/bin/LDASsaCN_mpi.x* Check the restart file: /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/input/restart > ls -l lrwxrwxrwx 1 fzeng g0620 76 2018-01-18 12:41 output -> /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_DE720_19/output/ cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/run qsub lenkf.0.j ========== 20180118: 1. Continued working on 4 on 20180117. 2. Made the maps of Princeton/MERRA2 ratio for: GPP, IPAR, APAR, IPAR/SWdown ~/Catchment/M2n5P_m0001/princetonVSmerra2/compare_princeton_merra2.m ~/Catchment/matlab/m_map/fz_plot_princeton_merra2_diff.m 3. Met with Randy and Eunjee. Randy suggests to do an experiment that uses MERRA-2 with SMAP N5 precipitation, and scales PARdir and PARdif using (1) the assumption of IPAR(i.e. vis)/SWdown=0.5 as in clm4_IPAR_DE720, and (2) the SWdown climatology from Qian et al., 2006 (or Princeton forcing if can't find Qian et al., 2006 dataset on Discover) to examine the effect of both the assumption and the difference in the SWdown data between original CLM4 and MERRA2 4. Modeling small group meeting: 3-4:30pm. ========== 20180119: 1. clm4_IPAR_DE720 crashed because the boundary condition directory in the driver namelist was still set to Sarith's nobackup. Corrected that and did an interactive run. It crashed due to floating invalid in L944 in process_cn.F90 that I modified: PARDIR(n) = met_force(n)%PARdrct * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) PARDIF(n) = met_force(n)%PARdffs * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) Should have added the if statement and make it become: if(met_force(n)%PARdrct+met_force(n)%PARdffs>0) then PARDIR(n) = met_force(n)%PARdrct * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) PARDIF(n) = met_force(n)%PARdffs * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) end if Corrected that and compiled. /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 > /bin/cp -pr Linux/ exec/clm4_IPAR/. It crashed again: floating divide by zero in L1758 catchmentCN.F90 "RHOAIR = PSUR(N) * 100. / (RGAS * TC(N))". Changed the block in process_cn.F90 to: if(met_force(n)%PARdrct+met_force(n)%PARdffs>0) then PARDIR(n) = met_force(n)%PARdrct * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) PARDIF(n) = met_force(n)%PARdffs * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) else PARDIR(n) = met_force(n)%PARdrct PARDIF(n) = met_force(n)%PARdffs end if Crashed again giving the same error message. Tried this in process_cn.F90: if(met_force(n)%PARdrct+met_force(n)%PARdffs>0 .and. met_force(n)%SWdown>0) then PARDIR(n) = met_force(n)%PARdrct * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) PARDIF(n) = met_force(n)%PARdffs * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) else PARDIR(n) = met_force(n)%PARdrct PARDIF(n) = met_force(n)%PARdffs end if Crashed. Tried this in process_cn.F90: PARDIR(n) = met_force(n)%PARdrct PARDIF(n) = met_force(n)%PARdffs ! PARdir and PARdif are available in MERRA-2. ! Use this modification to examine the impact of the assumption that ! half of SWdown is photosynthetically active (IPAR). ! fzeng, 17 Jan 2018 if(met_force(n)%PARdrct+met_force(n)%PARdffs>0 .and. met_force(n)%SWdown>0) then PARDIR(n) = met_force(n)%PARdrct * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) PARDIF(n) = met_force(n)%PARdffs * 0.5*met_force(n)%SWdown/(met_force(n)%PARdrct+met_force(n)%PARdffs) end if Crashed. Used the process_cn.F90 without any of these modifications. Still crashed at the same place. The clm4_DE720 run worked. Why does it crash now? -rw-r--r-- 1 fzeng g0620 71517 2017-08-10 14:47 process_cn.F90.restore -rw-r--r-- 1 fzeng g0620 48035 2017-08-10 16:12 lsm_calib_routines.F90.orig -rw-r--r-- 1 fzeng g0620 55883 2017-08-11 13:41 lsm_calib_routines.F90 -rw-r--r-- 1 fzeng g0620 80553 2017-11-03 15:25 tile_coord.F90 -rw-r--r-- 1 fzeng g0620 72098 2018-01-19 11:55 process_cn.F90.IPAR -rw-r--r-- 1 fzeng g0620 71517 2018-01-19 12:06 process_cn.F90 Here process_cn.F90 is identical to process_cn.F90.restore. The executable for the clm4_DE720 run: -rwxr-xr-x 1 fzeng g0620 69931463 2017-08-11 13:44 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/exec/clm4/Linux/bin/LDASsaCN_mpi.x* According to the time stamps, it looks like process_cn.F90.restore and lsm_calib_routines.F90 were used for the clm4_DE720 run. lsm_calib_routines.F90 has subroutine io_catchcn_rst but lsm_calib_routines.F90.orig doesn't. This subroutine is called from Applications/LDAS_App/lsm_calib_driver.F90. -rw-r--r-- 1 fzeng g0620 35447 2017-08-16 14:20 Applications/LDAS_App/lsm_calib_driver.F90 lsm_calib_driver.F90 is newer than the executable for the clm4_DE720 run, so it's not used in the the clm4_DE720 run. -rw-r--r-- 1 fzeng g0620 35447 2017-08-16 14:20 lsm_calib_driver.F90.6tiles -rw-r--r-- 1 fzeng g0620 25437 2017-08-10 15:00 lsm_calib_driver.F90.extract_forcing -rw-r--r-- 1 fzeng g0620 23808 2016-08-19 15:22 lsm_calib_driver.F90.NOsplitting -rw-r--r-- 1 fzeng g0620 23061 2016-08-19 11:04 lsm_calib_driver.F90.orig lsm_calib_driver.F90.NOsplitting is for a test run to see what will happen if we don't split the vegetation types in the 32-42 latitude bands. Not what we need here. lsm_calib_driver.F90.extract_forcing is for extracting forcing data for point runs. Not what we need here. lsm_calib_driver.F90.6tiles is for point runs. Not what we need here. Tried using lsm_calib_driver.F90.orig: /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/Applications/LDAS_App > cp lsm_calib_driver.F90.orig lsm_calib_driver.F90 Still crashed in the same place. Checked files newer than the executable: (1) GEOSlana_GridComp: -rw-r--r-- 1 fzeng g0620 80553 2017-11-03 15:25 tile_coord.F90 It's newer than the executable. The change I made then is adding this block below. This should not affect the run. case('CF0090x6C_DE0360xPE0180-Pfafstetter.til') date_line_on_center = .false. pole_on_center = .false. (2) GEOScatchCN_GridComp: all files are older than the executable. (3) GEOSsurface_GridComp/GEOSland_GridComp/Shared: -rw-r--r-- 1 fzeng g0620 5035 2017-11-02 16:52 catch_constants.mod -rw-r--r-- 1 fzeng g0620 7560 2017-11-02 16:52 catch_constants.o -rw-r--r-- 1 fzeng g0620 1035 2017-11-02 16:52 sibalb_coeff.mod -rw-r--r-- 1 fzeng g0620 17424 2017-11-02 16:52 sibalb_coeff.o -rw-r--r-- 1 fzeng g0620 99730 2017-11-02 16:52 lsm_routines.mod -rw-r--r-- 1 fzeng g0620 1198840 2017-11-02 16:53 lsm_routines.o -rw-r--r-- 1 fzeng g0620 1686 2017-11-02 16:53 update_model_paras.mod -rw-r--r-- 1 fzeng g0620 15896 2017-11-02 16:53 update_model_paras.o -rw-r--r-- 1 fzeng g0620 1241318 2017-11-02 16:53 libGEOS_LandShared.a These are from compiling the code. Could they affect the run? Need to re-compile? (4) Everything in GEOScatch_GridComp, GEOSroute_GridComp, and GEOSvegdyn_GridComp under GEOSsurface_GridComp/GEOSland_GridComp is older than the executable. (5) Applications/LDAS_App: -rw-r--r-- 1 fzeng g0620 35447 2017-08-16 14:20 lsm_calib_driver.F90.6tiles -rwxr-xr-x 1 fzeng g0620 144496 2017-11-03 15:18 cnlsm_ensdrv_main.F90* -rw-r--r-- 1 fzeng g0620 23061 2018-01-19 13:37 lsm_calib_driver.F90 lsm_calib_driver.F90 is the same as lsm_calib_driver.F90.orig and -rw-r--r-- 1 fzeng g0620 23061 2016-08-19 11:04 lsm_calib_driver.F90.orig What change did I make to cnlsm_ensdrv_main.F90 on 2017-11-03? This block of code is commented out in cnlsm_ensdrv_main.F90 compared to cnlsm_ensdrv_main.F90.SPLIT ( 2016-08-23 17:07 ) ! ! This is for updating tunable parameters - CLM4to4_5 upgrade ! call GetCalib_Resources(Nvar_Calib, model, expdomain, ITER, calib_data, calib_names) ! if ( NVar_Calib >= 1) then ! do i = 1, NVar_Calib ! if(trim(calib_names(i)) == 'SPLIT') & ! call upd_params ('SPLIT',calib_data(i,1)) ! end do ! endif ! End CLM4to4_5 upgrade So only this file below is newer than the executable AND may affect the run: Applications/LDAS_App/cnlsm_ensdrv_main.F90 Tried this: cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/Applications/LDAS_App mv cnlsm_ensdrv_main.F90 cnlsm_ensdrv_main.F90.20171103 cp cnlsm_ensdrv_main.F90.SPLIT cnlsm_ensdrv_main.F90 Compiled Applications/LDAS_App only. It again crashed in the same place. Tried "gmake clean" and "gmake install" to compile GEOSland_GridComp/Shared, GEOScatchCN_GridComp, GEOSlana_GridComp and Applications/LDAS_App. This time it crashed in L1749 catchmentCN.F90 "DHSDEA(N) = DHSDQA(N) * EPSILON / PSUR(N)" saying "floating divide by zero". Finally noticed that it's because the restart_id in lenkf.0.i is wrong!!! In lenkf.0.i: change "-restart_id clm4_IPAR_DE720 \" to "-restart_id clm4_DE720 \" Now it's running. Run one month and see if the output is identical to that in clm4_DE720_20. /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/post > diff clm4_IPAR_DE720_198001.gdat ../../clm4_DE720_20/post/clm4_DE720_198001.gdat It returned nothing, so they are identical. Great!!!!! Now: cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/Components/GEOSlana_GridComp cp process_cn.F90.IPAR process_cn.F90 gmake install cd ../../Applications/LDAS_App gmake install cd ../../.. /bin/cp -pr Linux/ exec/clm4_IPAR/. Do an interactive run. It's running. Run for one month and check if IPAR/SWdown is 0.5 on GrADS. Yes. IPAR/SWdown is 0.5 everywhere. And GPP is higher than clm4_DE720_20. 2. Met with Lesley and Eunjee about the Hindcasts forcing. Lesley will ask Andrea - if they can write out DRPAR and DRPAR in the output in the future, also write out hourly or 3 hourly output to represent diurnal cycle. Ben - how to use daily mean forcing in models. 3. For the experiment Randy suggested on 20180118, look at the Princeton data: The Princeton forcing dataset (v2) we were using was replaced by the latest version v2.2 in late April of 2017. See the email David forwarded to me titled "Fwd: Reminder: Update to Princeton Global Reanalysis forcing dataset". cd /discover/nobackup/fzeng/princeton mkdir v2_old mv * v2_old/ mkdir v2 cp v2_old/listx v2/. cd v2_old cp -p new.txt old.txt emails prcp_climatology.xdf ../v2/. cd ../v2 nedit listx listx [run listx to create the symbolic links] Read Greg's /home/gkwalker/Catchment/princeton_netcdf_cli.f90 to see how he computes Princeton precipitation climatology. ========== 20180122: 1. Computed global annual GPP and NEE from clm4.5_DE720_38. 2. Checked the CNNDynamicsMod.F90 in both CLM4 and CLM4.5: it doesn't have nitrogen flux from soil to the atmopshere. 3. Prepared for working from home due to government shutdown. 4. Processed the output of clm4_IPAR_DE720: ~/Catchment/CLM4 > grid_restore_hdeg clm4_IPAR_DE720 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4 > mv clm4_IPAR_DE720 clm4_IPAR_DE720_20 Used compare_ts.m in ~/Catchment/M2n5P_m0001/princetonVSmerra2 to make plots to compare clm4_IPAR_DE720_20 to clm4_DE720_20. Surprisingly, GPP in clm4_IPAR_DE720_20 is only about 1.2 PgC/yr higher than GPP in clm4_DE720_20. Is it because it's not in equilibrium yet (NEE = -0.42 PgC/yr as opposed to -0.13 PgC/yr in clm4_DE720_20). Checked and made sure that IPAR equals 0.5*SWdown in clm4_IPAR_DE720_20. 5. Set up and automate the clm4_IPAR_DE720 spinup run: (1) Create the template: cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/exec/clm4_IPAR/Linux/bin source /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/g5_modules ./ldsetup setup /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5_cn.exe /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5.bat --runmodel --monthsperjob 96 --landmodel catchCN cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/ mv clm4_IPAR_DE720 clm4_IPAR_DE720_template cd clm4_IPAR_DE720_template/run nedit lenkf.0.j & changed "-restart_path ../input/restart/rst_clm4 \" to "-restart_path ../input/restart/output \" cp -p ../../clm4_DE720_19/run/edit_lenkf.csh . cp -p ../../clm4_DE720_19/run/LSM_Calib.rc . mkdir ../lenkf_orig cp -p lenkf.*.j ../lenkf_orig/. ./edit_lenkf.csh Change the last line to "/home/fzeng/Catchment/CLM4/submit_next_batch_IPAR_DE720 clm4_IPAR_DE720" in lenkf.4.j (2) Set up the clm4_IPAR_DE720_21 cycle following steps in /home/fzeng/Catchment/CLM4/submit_next_batch_DE720: cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4 set exp = clm4_IPAR_DE720 set num = 20 Make sure the 20th cycle finished: ls -l $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4/${exp}_${num}/output/global/rs/ens0000/Y2017/M01/${exp}.ens0000.catchcn_ldas_rst.20170101_0000z cd ${exp}_${num}/output/global/rs/ens0000 /bin/rm -rf Y19?? Y200? Y201[0-6] mkdir -p Y1980/M01 cd Y1980/M01 ln -s ../../Y2017/M01/${exp}.ens0000.catchcn_ldas_rst.20170101_0000z ${exp}.ens0000.catchcn_ldas_rst.19800101_0000z cd $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4/${exp}_${num} /bin/rm -rf lenkf_orig cd $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4 /bin/cp -r ${exp}_template ${exp} cd ${exp}/input/restart ln -s $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4/${exp}_${num}/output /bin/rm rst_clm4 cd ../../run Check the executable: ls -l /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/build/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 69935568 2018-01-19 15:42 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/build/Linux/bin/LDASsaCN_mpi.x* Check the restart file: /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/input/restart > ls -l lrwxrwxrwx 1 fzeng g0620 81 2018-01-22 15:16 output -> /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720_20/output/ cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/run Did an interactive run to make sure it works: interactive.py -A sp3 -n 96 -a g0620 -X --debug cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/run ./lenkf.0.i It's running and read the right restart file. Stopped it. qsub lenkf.0.j 6. Modified process_cn.F90 so that the assumptions IPAR=0.5*SWdown and pardir=pardif are also used for MERRA-2 forcing. This is to examine the impact of these two assumptions combined. See /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/Component/GEOSlana_GridComp/process_cn.F90.IPAR.DReqDF. Compiled the code. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 mkdir -p exec/clm4_IPAR_DReqDF /bin/cp -pr Linux exec/clm4_IPAR_DReqDF/. 7. Set up and automate the clm4_IPAR_DReqDF_DE720 spinup run: (1) Create the template: Modified D0.5_cn.exe and D0.5.bat in /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/exec/clm4_IPAR_DReqDF/Linux/bin source /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/g5_modules ./ldsetup setup /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5_cn.exe /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5.bat --runmodel --monthsperjob 96 --landmodel catchCN cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/ mv clm4_IPAR_DReqDF_DE720 clm4_IPAR_DReqDF_DE720_template cd clm4_IPAR_DReqDF_DE720_template/run nedit lenkf.0.j & changed "-restart_path ../input/restart/rst_clm4 \" to "-restart_path ../input/restart/output \" cp -p ../../clm4_DE720_19/run/edit_lenkf.csh . cp -p ../../clm4_DE720_19/run/LSM_Calib.rc . mkdir ../lenkf_orig cp -p lenkf.*.j ../lenkf_orig/. ./edit_lenkf.csh Change the last line to "/home/fzeng/Catchment/CLM4/submit_next_batch_DE720 clm4_IPAR_DReqDF_DE720" in lenkf.4.j (2) Set up the clm4_IPAR_DReqDF_DE720_20 cycle following steps in /home/fzeng/Catchment/CLM4/submit_next_batch_DE720: set exp = clm4_IPAR_DReqDF_DE720 cd $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4 /bin/cp -r ${exp}_template ${exp} cd ${exp}/input/restart ln -s $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4/clm4_DE720_19/output /bin/rm rst_clm4 cd ../../run Check the executable: ls -l /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DReqDF_DE720/build/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 69931472 2018-01-22 16:05 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DReqDF_DE720/build/Linux/bin/LDASsaCN_mpi.x* Check the restart file: /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DReqDF_DE720/input/restart > ls -l lrwxrwxrwx 1 fzeng g0620 76 2018-01-22 16:31 output -> /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_DE720_19/output/ cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DReqDF_DE720/run nedit lenkf.0.j & change "-restart_id clm4_IPAR_DReqDF_DE720 \" to "-restart_id clm4_DE720 \" Did an interactive run to make sure it works: interactive.py -A sp3 -n 96 -a g0620 -X --debug cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DReqDF_DE720/run ./lenkf.0.i It's running and read the right restart file. Stopped it. qsub lenkf.0.j ========== 20180123: 1. New and old Princeton forcing: see David's email on 5/11/2017. 2. Double checked the land area used to compute global carbon fluxes for the Princeton run and the MERRA2 runs. My GPP is higher than Eunjee's (for both Princeton run and MERRA2 runs) because I use 390ppm CO2 concentration while Eunjee's is from the transient CO2 run (after pre-industrial spinup). 3. Set PEP (Progress and Engagement Program) goals on ADP. 4. Computed the global annual GPP and NEE in the 39th cycle of clm4.5_DE720. 5. Run clm4.5 on DDT to investigate why burned area is so high. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4.5/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 ls -l exec/clm4.5/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 71777037 2018-01-10 11:59 exec/clm4.5/Linux/bin/LDASsaCN_mpi.x* ls -l exec/clm4.5_debug/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 99760874 2017-12-27 13:16 exec/clm4.5_debug/Linux/bin/LDASsaCN_mpi.x* The executable in exec/clm4.5_debug needs to be updated. "gmake clean" and "gmake install BOPT=g" in GEOScatchCN_GridComp, GEOSlana_GridComp and Applications/LDAS_App. /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4.5/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 > /bin/cp -pr Linux exec/clm4.5_debug/. Set up the debug run following ~/Catchment/CLM4.5/submit_next_batch_DE720: cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5 /bin/cp -r clm4.5_DE720_template clm4.5_debug cd clm4.5_debug/input/restart ln -s $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720_39/output /bin/rm rst_clm4.5 cd ../../run nedit lenkf.0.j & (1) Change "/discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720/build/Linux/bin/LDASsaCN_mpi.x" to "/discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_debug/build/Linux/bin/LDASsaCN_mpi.x" (2) Change "-exp_id clm4.5_DE720 \" to "-exp_id clm4.5_debug \" nedit driver_inputs_0.5D_GLOBAL.nml & (identified the pixel with high burned area in Sahel using GrADS) minlon = -1. ! min longitude maxlon = -0.5 ! max longitude minlat = 17. ! min latitude maxlat = 17.5 ! max latitude cp lenkf.0.j lenkf.0.i nedit lenkf.0.i & (refer to /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_clm4fsun_point/run/lenkf.0.i) Make build point to the right executable: cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_debug rm -f build ln -s /gpfsm/dnb31/fzeng/clm4-to-clm4.5/LDAS/clm4.5/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/exec/clm4.5_debug build ls -l /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_debug/build/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 99751106 2018-01-23 15:10 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_debug/build/Linux/bin/LDASsaCN_mpi.x* Check the restart file: ls -l input/restart/ total 0 lrwxrwxrwx 1 fzeng g0620 80 2018-01-23 15:17 output -> /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720_39/output/ cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_debug/run ./lenkf.0.i Found a bug!! Vegetation types in this grid cell: 11: broadleaf deciduous temperate shrub [moisture stress only], fveg = 0.002 17: warm c4 grass [moisture stress only], fveg = 0.998 However, cropf_col (cropland frac.) = 0.998. Warm c4 grass is counted as crop incorrectly!! This is because of this block if( ivt(p) > nc4_grass )then cropf_col(c) = cropf_col(c) + wtcol(p) end if And nc4_grass = 16! I should have changed nc4_grass here to nc4_grass2 (=17) after I added the split types to pftvarcon.F90!! ========== 20180124: 1. Fixed the bug in CNFireMod.F90. Also checked all the other indices in pftvarcon.F90 to make sure they are used correctly. "gmake clean" and "gmake install BOPT=g" in GEOScatchCN_GridComp, GEOSlana_GridComp and Applications/LDAS_App. /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4.5/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 > /bin/cp -pr Linux exec/clm4.5_debug/. Run on DDT again. farea_burned is now 0 in all 3 columns for the first 3 CN time steps that I checked. Looks correct now. Wait until the current cycle to finish before recompiling the code. 2. Read the Oliphant and Stoy in press paper: partitioning of PAR to direct and diffuse components. 3. Compute 1980-2014 IPAR climatology for Princeston V2.2 and MERRA-2. For Princeton climatology: created ~/Catchment/princeton_monthly_cli.f90 following ~/Catchment/princeton_netcdf_cli.f90.gkw (same as /home/gkwalker/Catchment/princeton_netcdf_cli.f90). ========== 20180125: 1. Continued working on 3 on 20180124. For MERRA2 climatology: created ~/Catchment/merra2_monthly_cli.f90 following ~/Catchment/princeton_monthly_cli.f90 2. Met with Jana to help her with some questions about Catchment-CN. 3. Met with Randy, Lesley, Brad and Eunjee about the manuscripts. ========== 20180126: 1. Continued working on ~/Catchment/merra2_monthly_cli.f90. Trying to use the subroutines in subroutine get_GEOS in clsm_ensdrv_force_routines.F90. However, one subroutine calls another, making things very complicated. So far there are still these error messages when compiling it: /tmp/ifortReGtMm.o: In function `MAIN__': merra2_monthly_cli.f90:(.text+0xc38): undefined reference to `err_' /tmp/ifortReGtMm.o: In function `_unnamed_main$$_IP_gfio_getvar_': merra2_monthly_cli.f90:(.text+0x1e35): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x2013): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x207c): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x20a7): undefined reference to `getbegdatetime_' merra2_monthly_cli.f90:(.text+0x20c1): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x20e8): undefined reference to `diffdate_' merra2_monthly_cli.f90:(.text+0x2401): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x2889): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x28e2): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x29b4): undefined reference to `err_' merra2_monthly_cli.f90:(.text+0x29fd): undefined reference to `err_' /tmp/ifortReGtMm.o:merra2_monthly_cli.f90:(.text+0x2a46): more undefined references to `err_' follow /tmp/ifortReGtMm.o: In function `_unnamed_main$$_IP_gfio_getvar_': merra2_monthly_cli.f90:(.text+0x321b): undefined reference to `identifydim_' /tmp/ifortReGtMm.o: In function `_unnamed_main$$_IP_get_geos_gfio_openfile_': merra2_monthly_cli.f90:(.text+0x3cb2): undefined reference to `err_' Tried copying diffdate.f and gfio.f from /discover/nobackup/fzeng/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/GMAO_Shared/GMAO_gfio to ~/Catchment and adding these to subroutine GFIO_GetVar1: include "gfio.f" include "diffdate.f" But this caused more error messages, mostly due to the syntax errors caused by changes in FORTRAN version I guess. Tried to copy merra2_monthly_cli.f90 to /discover/nobackup/fzeng/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/Components/GEOSlana_GridComp to compile it, but don't know how to modify the GNUmakefile to make it generate the executable for merra2_monthly_cli.f90. The GNUmakefile here is different from the one in /discover/nobackup/fzeng/geos5_code/Icarus-1_0_p1/GEOSagcm/src/GEOSgcs_GridComp/GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp/GEOScatch_GridComp/mk_restarts. Ask Sarith next week (I think he comes back today or this weekend). 2. Monitored clm4.5_DE720: it's simulating Jan of 2008. Will check over the weekend. 3. Modified ~/Catchment/M2n5P_m0001/princetonVSmerra2/compare_ts.m to compare GPP and NEE among clm4_DE720, clm4_IPAR_DE720 and clm4_IPAR_DReqDF_DE720. Made plots comparing GPP and NEE time series among clm4_DE720_20 (in reasonable equilibrium), clm4_IPAR_DE720_21, and clm4_IPAR_DReqDF_DE720_20. ========== 20180128: 1. clm4.5_DE720 finished the 40th cycle. Checked CNFireMod.F90 to make sure that the bug about nc4_grass has been fixed. "gmake clean" and "gmake install" in GEOScatchCN_GridComp, GEOSlana_GridComp and Applications/LDAS_App. /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4.5/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 > /bin/cp -pr Linux exec/clm4.5/. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720 ls -l build/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 71777037 2018-01-28 10:52 build/Linux/bin/LDASsaCN_mpi.x* ls -l input/restart/ lrwxrwxrwx 1 fzeng g0620 86 2018-01-28 03:51 output -> /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4.5/clm4.5_DE720_21697831/output/ The executable is updated and the restart file is correct. cd run qsub lenkf.0.j 2. Both clm4_IPAR_DE720 and clm4_IPAR_DReqDF_DE720 stopped. Why? clm4_IPAR_DE720 stopped last night due to time out when it's simulating 20090916. Checked the time needed to simulate 8 years in the first 3 job segments (from 1980 to 2003, 8 years each)) that are finished. It took about 10.5 hours for each job segment. Don't know why this job segment took so long. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DE720/run ls -l ../output/global/rs/ens0000/Y2009/M09 -rw-r--r-- 1 fzeng g0620 291445863 2018-01-27 21:32 clm4_IPAR_DE720.ens0000.catchcn_ldas_rst.20090901_0000z This is the last restart file. cp lenkf.3.j lenkf.3b.j nedit lenkf.3b.j & [make it start from 20090901] qsub lenkf.3b.j clm4_IPAR_DReqDF_DE720 stopped yesterday afternoon again due to time out when it's simulating 20080615. Same as clm4_IPAR_DE720, the first 3 job segments took only about 10.5 hours each. Maybe Discover was too busy yesterday. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_IPAR_DReqDF_DE720/run ls -l ../output/global/rs/ens0000/Y2008/M06 -rw-r--r-- 1 fzeng g0620 291445863 2018-01-27 16:39 clm4_IPAR_DReqDF_DE720.ens0000.catchcn_ldas_rst.20080601_0000z This is the last restart file. cp lenkf.3.j lenkf.3b.j nedit lenkf.3b.j & [make it start from 20080601] qsub lenkf.3b.j 3. Computed the global annual GPP and NEE of clm4.5_DE720_40 using read_fields.m and calculate_global.m in ~/Catchment/matlab/clm4-to-clm4.5. GPP: 85.0712 NEE: -0.1845 ========== 20180129: 1. Updated higgins rain data through 28 Jan 2018. 2. Met with Eunjee to talk about her presentation in the IDS meeting on Wednesday. 3. Computed global annual GPP and NEE of clm4_IPAR_DE720_22 and clm4_IPAR_DReqDF_DE720_21. 4. Met with Randy, Rolf and Jana on Jana's calibration work. 5. Met with Randy and Eunjee on Eunjee's work. 6. About MERRA2 DRPAR and DFPAR monthly climatology, Qing told me that there are monthly average MERRA2 data files and I can just use them. Re-wrote ~/Catchment/merra2_monthly_cli.f90 and computed MERRA2 DRPAR and DFPAR monthly climatology. Also made plots of DRPAR/IPAR for the 12 months using ~/Catchment/merra2_DRPAR_DFPAR.gs. Sent the plots to Eunjee. ========== 20180130: 1. Modify the offline code for a simulation (clm4_pIPAR_DE720) that uses Princeton (v2.2) IPAR (assumming IPAR=0.5*SWdown) and MERRA2 IPAR (=DRPAR+DPAR) monthly climatology ratio to scale PARdir and PARdif in the run (uses MERRA2 forcing). See /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/Components/GEOSlana_GridComp/process_cn.F90.pIPAR Compiled: /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3 > /bin/cp -pr Linux exec/clm4_pIPAR/. 2. Set up and automate the clm4_pIPAR_DE720 spinup run: (1) Create the template: Modified D0.5_cn.exe and D0.5.bat in /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run. cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/exec/clm4_pIPAR/Linux/bin source /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/clm4/LDASsa_m3-16_0_p2_CatchCatchCN_for_MERRA3/src/g5_modules ./ldsetup setup /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5_cn.exe /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/run/D0.5.bat --runmodel --monthsperjob 96 --landmodel catchCN cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/ mv clm4_pIPAR_DE720 clm4_pIPAR_DE720_template cd clm4_pIPAR_DE720_template/run nedit lenkf.0.j & changed "-restart_path ../input/restart/rst_clm4 \" to "-restart_path ../input/restart/output \" cp -p ../../clm4_DE720_19/run/edit_lenkf.csh . cp -p ../../clm4_DE720_19/run/LSM_Calib.rc . mkdir ../lenkf_orig cp -p lenkf.*.j ../lenkf_orig/. ./edit_lenkf.csh Change the last line to "/home/fzeng/Catchment/CLM4/submit_next_batch_DE720 clm4_pIPAR_DE720" in lenkf.4.j (2) Set up the clm4_pIPAR_DE720_20 cycle following steps in /home/fzeng/Catchment/CLM4/submit_next_batch_DE720: set exp = clm4_pIPAR_DE720 cd $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4 /bin/cp -r ${exp}_template ${exp} cd ${exp}/input/restart ln -s $NOBACKUP/clm4-to-clm4.5/LDAS/tests/clm4/clm4_DE720_19/output /bin/rm rst_clm4 cd ../../run Check the executable: ls -l /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_pIPAR_DE720/build/Linux/bin/LDASsaCN_mpi.x -rwxr-xr-x 1 fzeng g0620 69947008 2018-01-30 11:41 /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_pIPAR_DE720/build/Linux/bin/LDASsaCN_mpi.x* Check the restart file: /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_pIPAR_DE720/input/restart > ls -l lrwxrwxrwx 1 fzeng g0620 76 2018-01-30 12:00 output -> /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_DE720_19/output/ cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_pIPAR_DE720/run nedit lenkf.0.j & change "-restart_id clm4_pIPAR_DE720 \" to "-restart_id clm4_DE720 \" Did an interactive run to make sure it works: interactive.py -A sp3 -n 96 -a g0620 -X --debug cd /discover/nobackup/fzeng/clm4-to-clm4.5/LDAS/tests/clm4/clm4_pIPAR_DE720/run ./lenkf.0.i It's running and read the right restart file. Stopped it. qsub lenkf.0.j 3. Modified ~/Catchment/matlab/m_map/fz_catchCN_obs.m to include 2007 Georgia drought. Made maps showing the anomalies. ========== 20180131: 1. Sarith presented the river routing model. 2. IDS meeting. 3. Meeting with Randy and Eunjee: update my work 4. 2008 Georgia drought analysis. 5. Hindcasts data: hourly Fortuna data is available in the archive, but hourly S2S v2.1 data is only available for a few fields for some periods. 6. Literature search for in situ FPAR measurements