The search begins

Ok, so now we need to...

1. Build a spreadsheet of stations in VA, data available at station, lat/long for stations
2. Gather ambient data for stations in VA
3. Convert lat/long to x/y and x/y into gridcell
4. Compare real data with model data
   

The yellow brick road is marked by these webpages:

• Ambient Data for Model Evaluation
  http://pah.cert.ucr.edu/vistas/ambient.shtml
  
• Ambient Datalist
  http://pah.cert.ucr.edu/vistas/docs/Ambient_datalist.doc
• Selected AQS Code Descriptions
  http://www.epa.gov/ttn/airs/airsaqs/manuals/codedescs.htm
• VIEWS Website
  http://vista.cira.colostate.edu/views/Web/IMPROVE/improve_data_prelim.htm
  

Let the hunt begin!

New PGroup license installed, CMAQ rebuilt

I've installed the new PGroup license file and rebuilt CMAQ. pgf90 is FAST, much faster than ifort or g95. (And before you say anything, I "make clean"'ed and deleted all .o and .mod before building). CMAQ is now ready to go. We just need the OCEAN file. From the output log:

>>--->> WARNING in subroutine OPEN3
File not available.


*** ERROR ABORT in subroutine RDEMIS_AE
Could not open OCEAN_1 file

Next steps

So, here's what we do next:

1) Benchmarkmark our results to make sure the data is good. Here's the plan:
a) Rebuild the model with commercial pgf90. Still need to get the license....
b) Rebuild with aero4. Need sea-salt OCEAN file. Linsey is finding this.
c) Edit the list of time-averaged output species (AVG_CONC_SPCS) in the run script.
d) Run the model with these species: NO2, NO, O3, HO, HO2, N2O5, HNO3, CO, ISOP, ETH, TOL, SO2, ASO4J, ASO4I, ANH4J, ANH4I, ANO3J, ANO3I, AORGAJ, AORGAI, AECJ, AECI
e) Compare our results to demo results.

2) Gather ambiant data for Virginia so we can compare model to real measurements. This could take some time, but here's a starting point: http://pah.cert.ucr.edu/vistas/ambient.shtml

3) Compare our results to real data. Here's some resources (not links, just files I'm supposed to already have somewhere) :
a) EVALUATION_TOOLS.txt
b) Model_Performance_Evaluation.pdf

(1)a-c shouldn't take more than a day. (1)d will take another 10+ hours of runtime. (1)e could take 1-2 days, depending on how complete we want to be.

(2) Might be a week or so, depending. It's just a lot of digging.

(3) ???? It's going to take some time to match real stations with grid cells and then extract the data. I'll know better when I get closer.

Did I miss anything?

CMAQ vistas is working

Rebuilding with pgf did the trick. Here's some notes:

1) Copied bldit.cctm.pgf to bldit.cctm.vistas
2) Modified bldit.cctm.vistas to match Linsey's settings. Here's the diff:

[jlinford@ene-411-lmarr cctm]$ diff bldit.cctm.pgf bldit.cctm.vistas
43c43
< set APPL = e1a
---
> set APPL = vistas
64,65c64,65
< #set ModDriver = ( module ctm $Revision; )
< set ModDriver = ( module ctm_yamo $Revision; )
---
> set ModDriver = ( module ctm $Revision; )
> #set ModDriver = ( module ctm_yamo $Revision; )
73,74c73,74
< #set ModInit = ( module init $Revision; )
< set ModInit = ( module init_yamo $Revision; )
---
> set ModInit = ( module init $Revision; )
> #set ModInit = ( module init_yamo $Revision; )
77,78c77,78
< #set ModAdjc = ( module denrate $Revision; )
< set ModAdjc = ( // yamo option does not need denrate )
---
> set ModAdjc = ( module denrate $Revision; )
> #set ModAdjc = ( // yamo option does not need denrate )
83,84c83,84
< #set ModHadv = ( module hppm $Revision; )
< set ModHadv = ( module hyamo $Revision; )
---
> set ModHadv = ( module hppm $Revision; )
> # set ModHadv = ( module hyamo $Revision; )
87,88c87,91
< #set ModVadv = ( module vppm $Revision; )
< set ModVadv = ( module vyamo $Revision; )
---
> set ModVadv = ( module vppm $Revision; )
> # set ModVadv = ( module vyamo $Revision; )
>
> #Use minimum vertical diffusivity of 1.0 m2/sec (FALSE) or new formulation (TRUE)
> set KZMIN = FALSE
110,111c113,114
< #set ModAero = ( module aero3 $Revision; )
< set ModAero = ( module aero4 $Revision; )
---
> set ModAero = ( module aero3 $Revision; )
> #set ModAero = ( module aero4 $Revision; )
117,118c120,121
< #set ModCloud = ( module cloud_radm $Revision; )
< set ModCloud = ( module cloud_acm $Revision; )
---
> set ModCloud = ( module cloud_radm $Revision; )
> #set ModCloud = ( module cloud_acm $Revision; )
128,129c131,132
< #set Mechanism = cb4_ae3_aq
< set Mechanism = cb4_ae4_aq
---
> set Mechanism = cb4_ae3_aq
> #set Mechanism = cb4_ae4_aq


And another diff:
[jlinford@ene-411-lmarr cctm]$ diff bldit.cctm.vistas_lmarr bldit.cctm.vistas
43d42
< #set APPL = e1a
53c52
< #set ParOpt # set for multiple PE's; comment out for single PE
---
> # set ParOpt # set for multiple PE's; comment out for single PE
65c64
< set ModDriver = ( module ctm $Revision; )
---
> set ModDriver = ( module ctm $Revision; )
74c73
< set ModInit = ( module init $Revision; )
---
> set ModInit = ( module init $Revision; )
78c77
< set ModAdjc = ( module denrate $Revision; )
---
> set ModAdjc = ( module denrate $Revision; )
84,85c83,84
< set ModHadv = ( module hppm $Revision; )
< #set ModHadv = ( module hyamo $Revision; )
---
> set ModHadv = ( module hppm $Revision; )
> # set ModHadv = ( module hyamo $Revision; )
88,89c87,88
< set ModVadv = ( module vppm $Revision; )
< #set ModVadv = ( module vyamo $Revision; )
---
> set ModVadv = ( module vppm $Revision; )
> # set ModVadv = ( module vyamo $Revision; )
91,92c90,91
< #Use minimum vertical diffusivity of 1.0 m2/sec (F) or new formulation (T)
< set KZMIN = F
---
> #Use minimum vertical diffusivity of 1.0 m2/sec (FALSE) or new formulation (TRUE)
> set KZMIN = FALSE
114c113
< set ModAero = ( module aero3 $Revision; )
---
> set ModAero = ( module aero3 $Revision; )
121c120
< set ModCloud = ( module cloud_radm $Revision; )
---
> set ModCloud = ( module cloud_radm $Revision; )
132c131
< set Mechanism = cb4_ae3_aq
---
> set Mechanism = cb4_ae3_aq
153c152
< set FC = /opt/intel/fc/9.0/bin/ifort
---
> set FC = /opt/pgi/linux86/6.2/bin/pgf90
157c156
< #set MPICH = /share/linux/bin/mpich-ch_p4
---
> set MPICH = /share/linux/bin/mpich-ch_p4
160,163c159,160
< #set FSTD = "-Mfixed -Mextend"
< #set F_FLAGS = "${FSTD} -O2 -module ${MODLOC} -I."
< #set F_FLAGS = "-extend_source 132 -cm -w95 -O2 -module ${MODLOC} -I. -c"
< set F_FLAGS = "-extend_source 132 -cm -w95 -O2 -module ${MODLOC} -I."
---
> set FSTD = "-Mfixed -Mextend"
> set F_FLAGS = "${FSTD} -O2 -module ${MODLOC} -I."
165,166c162
< #set C_FLAGS = "-v -O2 -I${MPICH}/include"
< set C_FLAGS = "-v -O2 -I."
---
> set C_FLAGS = "-v -O2 -I${MPICH}/include"
174c170
< set IOAPI = "${M3LIB}/ioapi_3/${BLD_OS}2_x86ifc -lioapi"
---
> set IOAPI = "${M3LIB}/ioapi_3/${BLD_OS}2_x86pg -lioapi"
206,207d201
< # LCM added the following line
< # cp -p ${WORK}/moddir/hgrd_defn.mod ${WORK}/moddir/vgrd_defn.mod $MODLOC

There are some differences in white space, but the content is the same (except for compiler, of course).

3) Run bldit.cctm.vistas: Complete. No errors.
4) Run nohup run.cctm.vistas: Still running. Past the sigfault. This could take a while. :)

Tutorial completed successfully on ene-411-lmarr with pgf

Yep, I managed to get through the entire tutorial without surgery on any code/scripts. Here's the complete log of how it's done:

TUTORIAL_PROCEDURE.txt:
Step 1)
README.txt:
Steps 1-6 complete.
Step 7)
IOAPI.txt:
Step 1-5 complete (export BIN=Linux2_x86pg).
Step 6) complete.
Step 8)
Skipped bldit.se.pgf
bldit.se_noop.pgf: complete.
Step 9)
Skipped
Step 10)
bldit.m3bld: complete.
Step 11)
bldit.jproc.pgf:
/usr/bin/ld: cannot find -lioapi
Forgot to link src/Linux2_x86pg. Fixed.
/usr/bin/ld: cannot find -lnetcdf
Forgot to build netCDF. Go back and do so...
CVS_NETCDF.txt:
(Building netCDF-3.6.0-p1 for Linux)
Steps 1-8 complete.
Fixed.
/opt/pgi/linux86/6.2/bin/pgf90 -Bstatic *.o -L/home/jlinford/models/jlinford/cmaq/lib/ioapi_3/Linux2_x86pg -lioapi -L/home/jlinford/models/jlinford/cmaq/lib/netCDF/Linux -lnetcdf -o JPROC_d1a
/home/jlinford/models/jlinford/cmaq/lib/ioapi_3/Linux2_x86pg/libioapi.a(rdatt3.o)(.text+0xa8a): In function `rdattc_':
: undefined reference to `nf_inq_att__'
/home/jlinford/models/jlinford/cmaq/lib/ioapi_3/Linux2_x86pg/libioapi.a(rdatt3.o)(.text+0xd71): In function `rdattc_':
: undefined reference to `nf_get_att_text__'
/home/jlinford/models/jlinford/cmaq/lib/ioapi_3/Linux2_x86pg/libioapi.a(rdatt3.o)(.text+0xe90): In function `rdattc_':
: undefined reference to `nf_get_att_int__'
..................... and many more ...........................

Try rebuilding ioapi_3 now that netCDF is in place...
IOAPI.txt:
Steps 1-5 complete (export BIN=Linux2_x86pg).
** NEW: Patched Makeinclude.Linux2_x86pg to use gcc as described.
Modified MXVARS3 to be 300 instead of 120 (just in case).
Step 6) complete.

bldit.jproc.pgf:
Completed successfully!
run.jproc 2>&1 | tee jproc.log:
Lots of errors about file permissions in /mnt/data/shenair/tutorial
chmod -R g+w /mnt/data/shenair/tutorial/*
run.jproc 2>&1 | tee jproc.log:
Complete. No errors in log file.
Step ICON)
bldit.icon.pgf: Complete. No errors.
run.icon 2>&1 | tee icon.log: Failed.
Errors from icon.log:
....
setenv INIT_CONC_1 /mnt/data/shenair/tutorial/icon/ICON_cb4_M_32_99TUT02_profile -v
....
*** ERROR ABORT in subroutine OPN_IC_FILE
Could not open nor create INIT_CONC_1 file
Make sure group is correct on tutorial directory:
sudo chgrp -R models /mnt/data/shenair/tutorial
run.icon 2>&1 | tee icon.log: Failed.
Errors from icon.log: Exactly the same as before
Remove old file ICON_cb4_M_32_99TUT02_profile:
rm /mnt/data/shenair/tutorial/icon/ICON_cb4_M_32_99TUT02_profile
run.icon 2>&1 | tee icon.log: Complete. No errors.
Step BCON)
bldit.bcon.pgf: Complete. No errors.
run.bcon 2>&1 | tee bcon.log: Failed.
Errors from bcon.log:
....
setenv BNDY_CONC_1 /mnt/data/shenair/tutorial/bcon/BCON_cb4_M_32_99TUT02_profile -v
....
*** ERROR ABORT in subroutine OPN_BC_FILE
Could not open nor create BNDY_CONC_1 file
Remove old file BCON_cb4_M_32_99TUT02_profile:
rm /mnt/data/shenair/tutorial/bcon/BCON_cb4_M_32_99TUT02_profile
run.bcon 2>&1 | tee bcon.log: Complete. No errors.
Step CCTM, Day 1)
Modified bldit.cctm.pgf and run.cctm for serial execution.
bldit.cctm.pgf: Complete. No errors.
run.cctm 2>&1 | tee cctm.log: Failed.
Errors from cctm.log:
/mnt/data/shenair/tutorial/cctm/CCTM_e1aCONC.e1a already exists
Remove old file
cd /mnt/data/shenair/tutorial/cctm/
mkdir jclbak
mv * jclbak
run.cctm 2>&1 | tee cctm.log: Complete. No errors.
Step CCTM, Day 2)
Modified run.cctm as described.
./run.cctm 2>&1 | tee cctm_e1b.log: Failed
Errors from cctm.log:
/mnt/data/shenair/tutorial/cctm/CCTM_e2aCONC.e2a not found
Need to replace all occurances of e2a with e1a and e2b with e1b becase we are not parallel.
./run.cctm 2>&1 | tee cctm_e1b.log: Complete. No errors.
Step ICON, rebuild)
Modified bldit.icon and run.icon as described (used CCTM_e1aCONC.e1a instead of CCTM_e2aCONC.e2a (serial execution)).
bldit.icon.pgf: Complete. No errors.
run.icon 2>&1 | tee icon.log: Complete. No errors.
Step BCON, rebuild)
Modified bldit.bcon as described.
bldit.bcon.pgf: Complete. No errors.
Modified run.bcon for Day 1 as described (used CCTM_e1aCONC.e1a instead of CCTM_e2aCONC.e2a (serial execution)).
run.bcon 2>&1 | tee bcon.log: Complete. No errors.
Modified run.bcon for Day 2 as described (used CCTM_e1aCONC.e1b instead of CCTM_e2aCONC.e2b (serial execution)).
run.bcon 2>&1 | tee bcon.log: Complete. No errors.
Step CCTM, rerun)
Modified run.cctm as described for day 1.
run.cctm 2>&1 | tee cctm.log: Complete. No errors.
Modified run.cctm as described for day 2.
run.cctm 2>&1 | tee cctm.log: Complete. No errors.

I'd just like to say....

THERE IS ABSOLUTELY NO REASON FOR THIS CONVOLUTED, COMPLICATED, UNSTABLE PROCESS! What's wrong with the computer science communitiy? This is not the first time I've had to drill holes in my head to get an air quality model to compile. And THIS ISN'T EVEN WITH MPI BUILT IN! Can't we design a simple build process for HPC applications?!?!? I'm going to have to answer that one, because it doesn't need to be this hard.

Things are going well.

Just to be extra careful, I deleted my copy of CMAQ and reinstalled fresh from the tgzs. I'm following the instructions very carefully, using pgf, and everything seems to work. One serious annoyance is the 5-second wait pgf enforces on their demo compiler. It's really slowing down compile times, but at least it's working. I guess they need to motivate sales somehow...

I've successfuly built and run ICON, BCON and CCTM. Took me days to get this far without pgf.

Portland Group Fortran

Downloaded and installed PGI Workstation 6.2 to /opt/pgi. Things seem to be much much smoother; I was able rebuild everything up to jproc without errors. Out of time now so I'll have to cary on later.

Worth it to get PGF?

Here's a thought: If we had Portland Group Fortran, CMAQ's default build scripts would light right up. Is it worth our time (money?) to get pgf? I'll see about finding a demo pgf or something to see if it makes me immediately more productive.

Could I find pgf in Dr. Sandu's lab perhaps?

Changing NetCDF flags breaks compilation on lmarr

I tried adding the -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE options to NetCDF build and now the icon, bcon and cctm models don't build. Still plugging away...

m3bld built on System X!

I've built m3bld on System X! It's actually kinda tricky.

1) Make the following changes to the build script:

sysx1:~/lmarr/cmaq/scripts/build jlinford$ diff bldit.m3bld.xlf bldit.m3bld
27,30c27,30
< # if ($BLD_OS != 'Linux') then
< # echo " $BLD_OS -> wrong bldit script for host\!"
< # exit 1
< # endif
---
> if ($BLD_OS != 'Linux') then
> echo " $BLD_OS -> wrong bldit script for host\!"
> exit 1
> endif
43,44c43,44
< set CC = /usr/bin/cc
< set CFLAGS = "-D__unix__"
---
> set CC = /usr/bin/gcc
> set CFLAGS = " "


2) Run the script to get the initial source. The build will fail.

3) Change the script again:

sysx1:~/lmarr/cmaq/scripts/build jlinford$ diff bldit.m3bld.xlf bldit.m3bld
27,30c27,30
< # if ($BLD_OS != 'Linux') then
< # echo " $BLD_OS -> wrong bldit script for host\!"
< # exit 1
< # endif
---
> if ($BLD_OS != 'Linux') then
> echo " $BLD_OS -> wrong bldit script for host\!"
> exit 1
> endif
43,44c43,44
< set CC = /usr/bin/cc
< set CFLAGS = "-D__unix__"
---
> set CC = /usr/bin/gcc
> set CFLAGS = " "
56,58c56,59
< # else
< # echo "Deleting files in $Bld"
< # /bin/rm $Bld/*
---
> else
> echo "Deleting files in $Bld"
> /bin/rm $Bld/*
> endif
60,61c61
< endif
< set echo
---
> #set echo
65,66c65,66
< #cvs export -r $Rel -d $Bld includes
< #cvs export -r $Rel -d $Bld m3bld
---
> cvs export -r $Rel -d $Bld includes
> cvs export -r $Rel -d $Bld m3bld


4) Modify BLD/bld_parser.c so lines 262-266 look like this:

#ifndef _WIN32
#ifndef __unix__
#include
#endif
#endif


(Or be clever and use && in the ifndif. Not sure why I didn't do that.....)

5) Run the build script again. It should work this time.

Pario built on System X!

I've built Pario after making the following changes to the build script:

sysx1:~/lmarr/cmaq/scripts/pario jlinford$ diff bldit.pario.xlf bldit.pario.pgf
28,31c28,31
< # if ($BLD_OS != 'Linux') then
< # echo " $BLD_OS -> wrong bldit script for host\!"
< # exit 1
< # endif
---
> if ($BLD_OS != 'Linux') then
> echo " $BLD_OS -> wrong bldit script for host\!"
> exit 1
> endif
41c41
< set MPI = /nfs/compilers/mpich-1.2.5/include
---
> set MPI = /share/linux/bin/mpich-ch_p4/include
45,46c45,46
< set FC = /opt/ibmcmp/xlf/8.1/bin/xlf95
< set FSTD = "-qfixed=110 -c"
---
> set FC = /share/linux/pgi/linux86/5.0/bin/pgf90
> set FSTD = "-Mfixed -Mextend -c"


I also had to create a link for the IOAPI_3 library in /nfs/home/jlinford/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3:


sysx1:~/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3 jlinford$ ls -l
total 8
lrwxr-xr-x 1 jlinford tcf 9 1 Oct 19:12 ioapi -> src/ioapi
drwxr-xr-x 13 jlinford tcf 442 1 Oct 17:53 src

Stenex built on System X!

I've built stenex with xlf after these changes made to the build script:

sysx1:~/lmarr/cmaq/scripts/stenex jlinford$ diff bldit.se.xlf bldit.se.pgf
27,30c27,30
< # if ($BLD_OS != 'Linux') then
< # echo " $BLD_OS -> wrong bldit script for host\!"
< # exit 1
< # endif
---
> if ($BLD_OS != 'Linux') then
> echo " $BLD_OS -> wrong bldit script for host\!"
> exit 1
> endif
43,44c43
< #set INCL = /share/linux/bin/mpich-ch_p4/include
< set INCL = /nfs/compilers/mpich-1.2.5/include
---
> set INCL = /share/linux/bin/mpich-ch_p4/include
47,49c46,47
< #set FC = /share/linux/pgi/linux86/5.0/bin/pgf90
< set FC = /opt/ibmcmp/xlf/8.1/bin/xlf95
< set FSTD = "-qfixed=110 -c"
---
> set FC = /share/linux/pgi/linux86/5.0/bin/pgf90
> set FSTD = "-Mfixed -Mextend -c"
111d108
< #echo "Skipping CVS export!"

IOAPI_3 built on System X!

I created the following Makeinclude file to get IOAPI to build with IBM xlf on System X:


sysx1:~/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3/src/ioapi jlinford$ cat Makeinclude.Darwin_ppcxlf
# @(#)$Header$
#
AR = ar
CC = cc
FC = xlf -w
M4 = m4
CPP = cpp
LINK = $(CC)

#MFLAGS = -tpp6 -traceback

OMPFLAGS =
OMPLIBS = -Vaxlib
COPTFLAGS = -O3 ${MFLAGS}
FOPTFLAGS = -O5 -qtune=auto -qarch=auth -qhot -qinitauto -qsave -qunroll ${MFLAGS}
FPPFLAGS =
FSFLAGS = -O5 -qtune=auto -qarch=auto -qhot -qinitauto -qsave
ARCHFLAGS = -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DAVOID_FLUSH=1

PARFLAGS =
ARCHLIB = -Bstatic -lc

M4DEFFILE = $(PVM_ROOT)/conf/$(PVM_ARCH).m4
PVMLIBS = -lpvm3


The library .o files are in /nfs/home/jlinford/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3/src/Darwin_ppcxlf


sysx1:~/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3/src/Darwin_ppcxlf jlinford$ pwd
/nfs/home/jlinford/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3/src/Darwin_ppcxlf
sysx1:~/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3/src/Darwin_ppcxlf jlinford$ ls
bilin.o currstep.o findc.o getstr.o julian.o locatsc.o opnlist3.o rdtflag.o sortis.o wrbndary.o
bmatvec.o currstepc.o findr1.o getyn.o julianc.o lustr.o opnlog3.o rdvars.o str2dble.o wrbuf3.o
bufint3.o daymon.o findr2.o grdchk3.o junit.o m3abort.o pcoef.o read3.o str2int.o wrcustom.o
cbarnes1.o daymonc.o findr3.o gridops.o kfindx.o m3err.o pgrdsum.o read3c.o str2real.o wrdict3.o
cbarnesN.o dble2real.o findr4.o hhmmss.o kfopen.o m3errc.o pmatvec.o read4d.o strlist.o wrgrdded.o
check3.o ddtvar3.o findsc.o hhmmssc.o kfread.o m3exit.o poly.o read4dc.o sync3.o wrgrnest.o
check3c.o ddtvar3c.o flush3.o index1.o kfwrite.o m3exitc.o promptdfile.o readsmet.o sync3c.o wriddata.o
chkfil3.o desc3.o gcd.o init3.o lambert.o m3mesgc.o promptffile.o realist.o syncfid.o write3.o
ckdesc3.o desc3c.o gctp.o init3c.o lblank.o m3msg2.o promptgrid.o rmfile.o synchtao.o write3c.o
ckfile3.o dmatvec.o get_endian.o initblk3.o len2.o m3warn.o promptmfile.o scanint.o systemf.o write4d.o
ckgeom.o dscgrid.o getdate.o initlog3.o libioapi.a m3warnc.o rdatt3.o sec2time.o time2sec.o write4dc.o
ckname.o dscgridc.o getdble.o inqatt3.o ll2utm.o mmddyy.o rdatt3c.o sec2timec.o time2secc.o wrpatch.o
cktflag3.o dt2str.o getdfile.o inqatt3c.o locat1.o mmddyyc.o rdbndary.o secsdiff.o trimlen.o wrprofil.o
close3.o dt2strc.o getdfilec.o interp3.o locat2.o name2fid.o rdbuf3.o secsdiffc.o ungridb.o wrsmatrx.o
close3c.o envgets.o getdttime.o interp3c.o locat3.o nameval.o rdcustom.o setsphere.o ungridi.o wrtflag.o
crdict3.o filchk3.o getefile.o intg2real.o locat4.o nextime.o rddict3.o shut3.o upcase.o wrvars.o
crlf.o filchk3c.o getefilec.o intlist.o locatc.o nextimec.o rdgrdded.o shut3c.o utm2ll.o xtbuf3.o
crtbuf3.o find1.o getffile.o iobin3.o locatr1.o open3.o rdgrnest.o skipl.o wkday.o xtract3.o
crtfil3.o find2.o getmenu.o ioparms3.o locatr2.o open3c.o rdiddata.o smatvec.o wkdayc.o xtract3c.o
crtkf.o find3.o getnum.o isdstime.o locatr3.o opnfil3.o rdprofil.o sortic.o wratt3.o year4.o
currec.o find4.o getreal.o jstep3.o locatr4.o opnkf.o rdsmatrx.o sortir.o wratt3c.o yr2day.o
sysx1:~/lmarr/cmaq/cmaq-4.5.1/lib/ioapi_3/src/Darwin_ppcxlf jlinford$

Built NetCDF on System X!

Using the script I found here: http://lists.apple.com/archives/Fortran-dev/2005/Jan/msg00012.html I successfully built NetCDF! All tests passed for both Fortran and C++ libraries so I guess it actualy works. Now, on to IOAPI!