====== Overview ======
Collegenet offers nightly downloads of new incoming applicants for the UMBC Graduate School. CollegeNet makes two separate text file that contains the metadata on each applicant, and multiple PDF files of their applications and recommendations.

One step of the UMBC Grad Schools [[collegenet_to_paradocs_import_process|processing of this data]], is to download them from collegenet. Automation of this step has been put into place on the [[:systems/paradocs/server_upgrade|Paradocs]] Linux machine by Nick Yeates, and is documented below.

====== Location ======
The shell script that does the actual automatic download from Collegenets servers to our AD I:/ drive is located on the [[:systems/paradocs/server_upgrade|paradocs server]] at 
  * <code>/var/paradocs/cnet2umbc/collegenet_download.sh</code>


This shell script is run by a crontab file located at
  * <code>/etc/cron.d/cnet2umbc</code>

====== Time ======
The download is run nightly by the paradocs cron job, at 5am EST in the morning, after collegenet releases the new daily data files.

**Note:** Around December 2007, it was found that collegenet was posting their data files AFTER UMBC had set to receive the data files. A three time-zone difference also led to an unnoticed time distortion. They release around 1:30 PST, which is 4:30 EST. This led to UMBC not automatically getting some data files overnight. It was fixed by altering the time in the crontab file. 

====== Logging / Debugging ======
All daily runs are extensively logged. You can look back many days to see what went bad.

/etc/cron.d/cnet2umbc.err.out

===== Explanation of how logging is implemented =====
    * Output by the crontab file located at /etc/cron.d/cnet2umbc , containing this code
      * <code>
##################################################################################################

# UMBC collegenet download; the following script downloads daily collegenet data files
# to UMBC AD department drive
00 5 * * * root sh /var/paradocs/cnet2umbc/collegenet_download.sh >> /etc/cron.d/cnet2umbc.err.out 2>&1

##################################################################################################</code>
      * The >> above appends the output (stdout) of the command, to the text file /etc/cron.d/cnet2umbc.err.out
      * 2>&1 reroutes the 2nd output stream (error messages or stderr) to the 1st output stream (standard out or stdout). Now, both stdout AND stderr will go to the aforementioned text file!
    * the .sh file is a shell script file that contains commands such as
      * <code>
echo "====== SCRIPT RUN ON ======"
echo $(date)
echo -e "====== SCRIPT RUN ON ======\n\n"</code>


====== Mounting of UMBC I:/ drive in Linux ======
important shell commands on how to mount the gradschool I:/ drive (OIT AD drive) to the paradocs machine via SMB.

FIXME Need to fix so that the mount reloads on boot or on any loss of connection. Add to fstab or mtab?

<code shell>
  927  smbmount \\\\sharedvol.ad.umbc.edu\\dept /mnt/grads/ -o username=ad\\nyeates1
  928  mkdir /mnt/grads/
  929  smbmount \\\\sharedvol.ad.umbc.edu\\dept /mnt/grads/ -o username=ad\\nyeates1
  930  cd /mnt/grads/GradS/
  931  ls -la
  932  cd CollegeNetData/
  933  ls -la
  934  pwd
  935  man smbumount
  936  smbumount /mnt/grads/
  937  cd /var/paradocs/
  938  smbumount /mnt/grads/
  943  cat /etc/mtab
  944  cat /etc/fstab
  945  cat /etc/mtab
  946  cat /etc/mtab
  947  cat /etc/fstab cnet2umbc.err.out 
  948  cat cnet2umbc
  949  nano cnet2umbc

**BEST SMBMOUNT COMMAND**
smbmount \\\\sharedvol.ad.umbc.edu\\dept /mnt/sharedvol.ad.umbc.edu/dept/ -o username=ad\\nyeates1
</code>