Robotics Server

The operating sytem on the server is 64-bit Debian Jessie and it is running on a HP xw4600 with machine hardwared name FRC 2011-6.  It was put into service December 2016.  There are notes for how that machine was built and configured in Michael's document files.

We use to a "CompuLab fit-PC2 Value (Rev 1.4), Atom Z510 1.1 GHz, RAM 1 GB, bay for 2.5" SATA HDD" that we paid $275.00 to purchase from Amazon.com.  Fit-PC website revisions page, BIOS ROM files, support page, and reference guide.  There is also a copy of the reference guide on the team Goolge Docs site under Tools and Machines.  It is running Debian Wheezy and was stood up on January 3, 2014, the day before the FRC season kickoff.  It was replaced late December 2016.

The prior machine was a GuruPlug Server Plus plug computer purchased from GlobalScale Technologies.  It has a USB hard disk attached to it.  It is running Debian Lenny and needs to be updated to Squeeze.

Disk Configuration

The following directories in / are used by SVN, the Apache web server, and for backups.

  • svn-repos: svn repositories
  • backups: Backups made to this machine from this machine and remote machines by commands like tar and cpio.  This the primary version (source location) of these backups.  They can and should be duplicated to other machines using commands like rsync.  A prime candidate is helium at the Schuh home.
  • backupsSecondary:  We use this to make off site backups of backup archives like the ones we keep on helium and iron at the Schuh home.  Be sure and put them in subdirectories based on the machine name.
  • backupsNoBackup: Used to put files on that are not to be backed up to any other machine.  They can be deleted but are being kept around just in case something goes wrong and they save the day.  For example, old versions of svn backups.
  • /www: All web pages accessed from http://robotics.mvla.net/{files,tmp} are in /www/http and all web pages accessed from https://robotics.mvla.net/{files,tmp} are in /www/https.  Within these, we put season specific files in directories like http://robotics.mvla.net/files/2012/images and  http://robotics.mvla.net/files/2012/videos.
    • The /etc/apache2/sites-available/robotics is used to control permissions https pages.  Restart with /etc/init.d/apache2 restart.
    • To facilitate people adding files to the http and https websites, we created symbolic links from the user's home directory to the appropriate directories.  i.e. for Jerry
         ls -l ~jerry
         lrwxrwxrwx 1 root  root    27 2012-02-23 21:14 https_files -> /www/https/files/2012/jerry
         lrwxrwxrwx 1 root  root    26 2012-02-23 21:27 www -> /www/http/files/2012/jerry
      This way, jerry, dave, parker, austin, and michael can quickly add files to the http and https parts of the website.  These users can also go to /www/http and /www/https and add files as needed because they are all in the webfiles group.  Michael and Dave

Disk Mounts

The external hard disk is mounted on /media/usb0.  To avoid having configuration files and symbolic links pointing /media/usb0 they are remounted to directories in /.  The subversion repository and scripts are mounted from the external hard disk on /medial/usb0 and this is remounted with the mount bind option to put the repository in /var/svn-repos.  Similarly, backups, backupsSecondary, backupsNoBackup, and www are remounted in /.

robotics:/media/usb0/svn-repos# df -h

Filesystem            Size  Used Avail Use% Mounted on
tmpfs                 251M     0  251M   0% /lib/init/rw
udev                   10M  744K  9.3M   8% /dev
tmpfs                 251M  4.0K  251M   1% /dev/shm
rootfs                463M  380M   84M  82% /
tmpfs                 251M   21M  231M   9% /var/cache/apt
/dev/sdc1             147G   36G  104G  26% /media/usb0

robotics:/media/usb0/svn-repos# mount
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
rootfs on / type rootfs (rw)
tmpfs on /var/cache/apt type tmpfs (rw,noexec,nosuid)
/dev/sdc1 on /media/usb0 type ext3 (rw)
/media/usb0/backups on /backups type none (rw,bind)
/media/usb0/backupsSecondary on /backupsSecondary type none (rw,bind)
/media/usb0/svn-repos on /var/svn-repos type none (rw,bind)
/media/usb0/backupsNoBackup on /backupsNoBackup type none (rw,bind)
/media/usb0/www on /www type none (rw,bind)
swap file - added Sept 4, 2016 after server ran out of memory and killed mysql and brought down frc971.org.
Followed directions at this website.  In short, I did:
dd if=/dev/zero of=/swapfile1 bs=1024 count=524288
chmod 0600 /swapfile1
mkswap /swapfile1
swapon /swapfile1
Add the following to /etc/fstab
/swapfile1 none swap sw 0 0

Unattended Upgrades

Used to keep the package list and packages upgraded all of time.

apt-get install unattended-upgrades

SVN

The /var/svn-repos/authz file is used to control access to the repository.  Look in the /var/log/apache2 directory for helpful log messages.  Use the following command to add a password
      htpasswd -s /etc/apache2/dav_svn.passwd   username

Gerrit

Aug 2, 2014 Brian proposed using Gerrit for managing the team's GIT repository and code reviews.  Michael and Brian decided to install the software and see if we could get it all working.  Gerrit documentation.  This is what we did.

  • We used the Gerrit for Debian GNU/Linux package builder to make a debian package that correctly locates Gerrit.
    • mkdir /seagate/tmp/gerrit_debian_package
    • cd !$
    • git clone https://github.com/dnaeon/gerrit-debian.git # made a copy of package sources
    • apt-get update
    • apt-get upgrade
    • apt-get install  build-essential git-core openjdk-7-jre-headless debhelper
    • cd /seagate/tmp/gerrit_debian_package/gerrit-debian
    • Fetched the most recent version of the gerrit.war file (this is a Java web server archive or something like that) from Gerrit Code Review - Releases.  We downloaded Gerrit 2.9 with: cd /seagate/tmp/gerrit_debian_package/gerrit-debian/src ; wget -O gerrit.war http://gerrit-releases.storage.googleapis.com/gerrit-2.9.war
    • Update the version numbers in debian/changelog by duplicating the first stanza and putting it at the top and changing the version number to 2.9 (the same as the download) and updating the author's name on line 5.
    • Update the java version number to be openjdk-7-jre-headless in debian/control
    • Build the package with: 
    • dpkg-buildpackage -us -uc  # this makes a set of install files in /seagate/tmp/gerrit_debian_package
      • -rw-r--r-- 1 root root  41365624 Aug  2 19:07 gerrit_2.9_all.deb
      • -rw-r--r-- 1 root root       590 Aug  2 19:06 gerrit_2.9.dsc
      • -rw-r--r-- 1 root root      1199 Aug  2 19:07 gerrit_2.9_i386.changes
      • -rw-r--r-- 1 root root 167028008 Aug  2 19:06 gerrit_2.9.tar.gz
    • cd .. ; dpkg -i gerrit_2.9_all.deb  # this creates a series of questions.  We used most of the defaults and set a few custom values.  The install hung on starting gerrit.  We had to kill off processes 8846 and 8797 to get the install to finish.  dpkg -l showed gerrit 2.9 as installed.
      • gerrit    8797  8794  2 19:15 ?        00:00:17 /usr/bin/java -jar /usr/share/gerrit/gerrit.war init -d /var/lib/gerrit/review_site
      • gerrit    8846     1 12 19:19 ?        00:00:43 GerritCodeReview -jar /var/lib/gerrit/review_site/bin/gerrit.war daemon -d /var/lib/gerrit/review_site --run-id=1407032397.8820
    • We will have to set up apache to properly deal with gerrit so that we can make more progress.
Web Configuration for Gerrit
  • cd /etc/apache2/
  • a2enmod proxy_http # enables http proxy module
  • AllowEncodedSlashes On
  • ProxyPass /gerrit/ http://127.0.0.1:8081/gerrit/ nocanon
    http://gerrit-review.googlesource.com/Documentation/config-reverseproxy.html
    ProxyRequests Off
    ProxyVia Off
    ProxyPreserveHost On
  • service apache2 restart  # Restart the server and it worked fine.  
  • Access gerrit by going to https://robotics.mvla.net/gerrit
  • Brian added some lines to the end of /var/lib/gerrit/review_site/etc/gerrit.config and restarted it.
    21a22,32
    > [download]
    >       scheme = ssh
    >       archive = tbz2
    >       archive = tgz
    > [gc]
    >       startTime = 4:00
    >       interval = 1 week
    > [plugins]
    >       allowRemoteAdmin = true
    > [suggest]
    >       from = 1
Starting and stopping Gerrit
  • We tried using "/etc/init.d/gerrit restart" to restart gerrit and it "failed".  We found that running it twice worked.
  • Restarting with "/etc/init.d/gerrit stop" followed by "/etc/init.d/gerrit start" worked.
More Gerrit setup and configuration
  • One thing that I think would be good for somebody else to do is to write a program to handle automatically ensuring every commit builds and the tests pass. It's not hard, but it is a bit of code to write. I think Python is the best language to use for it, but pretty much any will work. The basic idea is to use gerrit stream-events to watch for new patch sets being added and then use gerrit review to approve it or not and send a link to the results. If anybody is interested in working on that, I'd be happy to help.  (From Aug 6, 2014 email to Daniel from Brian)

GIT

To set up a git user, first set up an svn account.

  • vi /var/svn-repos/authz    # put in an entry listing who the person is and add them to the 971 group.
  • htpasswd -s /etc/apache2/dav_svn.passwd javier

Make a regular user account on robotics.mvla.net.  Ask the user to use a strong password.

  • adduser javier # adduser asks for the password first and then the user information (name etc).

Put the new git user in the webfiles group with the others.

  • vi /etc/group

Wrote /etc/apache2/make_git.passwd to create /etc/apache2/git{114,971}.passwd files using the team groups in /var/svn-repos/authz and the passwords from /etc/apache2/dav_svn.passwd.

  • /etc/apache2/make_git.passwd   # Run this so that git web works for the new user.

Quick usage guide for how to use git to track local files.  Hopefully, Michael can use these to replace rcs usage.  At least, that is Austin's hope.  Dec 25, 2015

# Create a repostory.  First cd into the dirctory that has the files to be revisioned.

cd /tmp
mkdir etc_test
cd etc_test
cp /etc/hosts . 

# Create a repository
git init

# Add a  file
git add hosts

# Check the status
git status

# Commit the file
git commit
git commit -m "Added Google hostname." hosts

# Show the versions with 
git log

michael@radon:/tmp/git_work$ git log

commit 1ec6ff291297bfc2192cb1e4c4e121214c491619
Author: Michael Schuh <michael@xx.com>
Date:   Fri Dec 25 11:49:44 2015 -0800

    Added Google hostname.

commit 009f1f0fb3200dc812ae5feb5d19cec859a5cf91
Author: Michael Schuh <michael@xx.com>
Date:   Fri Dec 25 11:38:06 2015 -0800

    Small change.

commit c115f9b887782170dcec3c20d4ecfc4877838f2b
Author: Michael Schuh <michael@xx>
Date:   Fri Dec 25 11:33:53 2015 -0800

    First version.

# Show recent activity with the commit actions with more detail by using the "reflog" command
git reflog 
009f1f0 HEAD@{0}: commit: Small change.
c115f9b HEAD@{1}: commit (amend): First version.
8c2d84d HEAD@{2}: commit (initial): First version.

# Compare with a particular commit with a uniq fraction of the number.
 
git diff 009f
# or with
 git diff 009f1f0fb3200dc812ae5feb5d19cec859a5cf91

# "master" is the term git users us for the equivalent of the svn "head".  It is the "branch" that everyone works on.

Gitiles

Gitiles is used to better view the team source code.

Clone the repository with

cd ~michael/src (I did this logged in as user michael)
git clone https://gerrit.googlesource.com/gitiles
cd gitiles
git checkout v0.2-5
# Use the build stand alone directions in https://gerrit.googlesource.com/plugins/gitiles/+/master/src/main/resources/+Documentation/build.md#Build-in-Gerrit-tree
# Copy 971-Robot-Code/tools/bazel ~/src/gitiles/tools
bazel build :gitiles  (This should build the code.)
# To install, follow the direction in
https://gerrit-review.googlesource.com/Documentation/cmd-plugin-install.html
# we had Parker run this command 
ssh -p 29418 localhost gerrit plugin install -n gitiles.war /home/michael/src/gitiles/bazel-bin/gitiles.war
This did not work either.  Brian said we have to build it in the gerrit software.  He said to use gerrit tag version 2.13.4.

Install the plugin following the directions at https://gerrit-review.googlesource.com/Documentation/cmd-plugin-install.html

 

Mail/qmail

On May 12, 2014, Austin installed qmail.  To make it work, he had to edit /etc/hosts and make sure that the 192.168.10.61 line had robotics.mvla.net as the first entry so that hostname -f would return the full host name.  He then installed qmail and qmail-run.

apt-get install qmail qmail-run

When he did this, apt-get install complained about the installed email package and asked that it be removed.  He probably removed it with "dpkg -r packageName".

He said that after that, everything just worked.

On May 12, 2014, Austin tried to get email working on robotics.mvla.net and left the root email configured in a way that email to root did not work.  Michael & Brian debugged this Aug 2, 2014.

  • Updated the root: entry in /etc/aliases to michael and forces the aliases file to be reloaded by running "newaliases".
  • Removed the /var/lib/qmail/.qmail-root file.  It pointed to "&alias" which did not exist.
  • Noticed that .qmail-postmaster pointed to "&alias" and this failed due to no "alias" user.  We fixed this by adding .qmail-alias file and putting "&root" in it.  qmail was restarted with the "qmailctl restart" command.
  • Added ~michael/.qmailrc with "|forward michael at boardsailor.com"
  • Now root, postmaster, and a bunch of other emails all go to Michael's Gmail account.

On Aug 3, 2014, Michael figured out that email to frc971.org was not being delivered.  Having fixed email to root the day before with Brian, Michael found error messages in his email saying email was bouncing.  The main complaint was that site at SpartanRobotics.org did not exist.  Because of this he was not receiving messages saying that there was out of date server software.  He also figured out that email sent from the team website contact page was not going through.  He fixed this by reading the Multiple host names section in the qmail man pages and then:

  • cd /var/lib/qmail/control (this is linked to /etc/qmail on our machine.
  • added frc971.org and spartanrobotics.schuhusa.org, and spartanrobotics.org on separate lines to both the locals and rcpthosts files.  They both contain:
    robotics.mvla.net
    frc971.org
    spartanrobotics.schuhusa.org
    spartanrobotics.org
  • Restarted qmail with: qmailctl reload
  • Added an alias so that email to site goes to Michael by adding the line "site: michael" to the bottom of the /etc/aliases file.
  • Made the mail system reread and use the updated /etc/aliases file by running "newaliases".  See the newaliases man page (man 1 newaliases) for more information on it.

On July 25, 2015, I added losaltosrobotics.org using the same procedure Brian and I used to add other names on August 3, 2014.

Backup Strategy

Cron jobs run by root backup the svn repository to /backups/svn/robotics.mvla.net.daily using the scripts in /seagate/svn-local-bin.

-rwxr-xr-x 1 root root 4480 2011-12-26 18:18 backupOnlyChangedArchives.csh
-rwxr-xr-x 1 root root 4725 2011-12-26 21:05 backupOnlyChangedArchivesDaily.csh
-rwxr-xr-x 1 root root 4185 2011-12-26 21:06 backupOnlyChangedArchivesWeekly.csh

Use the backupScripts.csh in /seagate/svn-local-bin to backup the scripts, notes, and configuration files to /backups/linux/robotics.mvla.net/ with names like 20111203.2206_subversion_notesAndConfig.tgz.  The backupScripts.csh script should be run by hand after changes to these files.

OS backups are created by a cron job that runs the  $HOME/bin/backupSchuhComputers.rb script with -0, -1, and -2 flags for full backup, weekly, and daily backups.  The backup files are saved in /backups/linux/robotics.mvla.net.  They are copied daily to helium by a cron job run from helium at Michael's home.  The /etc/backupSchuhComputers.exclude 

  • var/svn-repos
  • var/svn-repos.firstTry
  • usr/local/Plone-4.3.3
  • usr/local/Plone-4.3.3-UnifiedInstaller
  • var/log/auth.log  # Added because it would change as it was being backed up and cause the backup to fail.

The LosAltosRobotics.org website is restored by following the directions in /usr/local/Plone_notes.txt.  After reinstalling Plone from the Plone download page, be sure and add in all of the security patches listed on the Plone HotFixes page.

The FRC971.org Plone and robotics.mvla.net websites are restored as part of the linix OS backups.  The FRC971.org Plone database is also backed up seperately.

The /var/svn-repos directory is restored by pulling the var/svn-repos directory out of the 20111203.2206_subversion_notesAndConfig.tgz backup file generated by the backupScripts.csh script in /seagate/svn-local-bin directory.  The repositories are backed up in helium:/backups/SpartanRobotics/rsync/backups/svn/robotics.mvla.net.daily

 time tar -xzf /tmp/frc971-7770.tar.gz  # Commands to restore a repository from the tar and dump backups.
 svnadmin load frc971-7770 < /tmp/frc971_7771-7868.WeeklyIncr.dump
 svnadmin load frc971-7770 < /tmp/frc971_7869-7873.DailyIncr.dump
 svnadmin load frc971-7770 < /tmp/frc971_7874-7874.DailyIncr.dump
 svnadmin load frc971-7770 < /tmp/frc971_7875-7883.DailyIncr.dump
 svnadmin load frc971-7770 <  /tmp/frc971_7884-7887.DailyIncr.dump
 mv  
 frc971-7770 frc971
 chown -R www-data.subversion frc971

The external Seagate hard disk has the following contents:

drwxr-xr-x 5 root root  4096 Dec 31  2011 backups
drwxr-xr-x 2 root root  4096 Apr 14  2015 backupsNoBackup
drwxr-xr-x 2 root root  4096 Oct  7  2011 backupsSecondary
drwxr-xr-x 2 root root  4096 Dec 26 14:46 RCS
-rw-r--r-- 1 root root  1601 Dec 26 14:45 README.txt
drwxr-xr-x 3 root root  4096 Dec 26 14:48 svn-local-bin
drwxr-xr-x 4 root root  4096 Oct  6 20:58 tmp
drwxr-xr-x 4 root root  4096 Feb 23  2012 www

The RCS and README.txt files and svn-local-bin directory are backed up in the 20111203.2206_subversion_notesAndConfig.tgz backup file generated by the backupScripts.csh script in /seagate/svn-local-bin directory.  The backups and www directories are rsync'ed to helium nightly and the others are not backed up.  The README.txt describes their contents.

Cron jobs on helium (plug computer located at the Schuh home) run under the michael user to backup the /backups/ directory to helium:/backups/SpartanRobotics/backups_from_robotics.mvla.net_via_rsync .

# min hour dayOfMonth month dayOfWeek command
#
# ===========
# =========== Backup robotics.mvla.net:/backups
# ===========     Run it at 4 am.
    0      4     *          *         *       (echo -e "\n============ "; date ; rsync -av  --log-file=$HOME/log/rsync_robotics.log   michael@robotics.mvla.net:/backups /backups/SpartanRobotics/backups_from_robotics.mvla.net_via_rsync ) >> $HOME/log/rsync_stdo_robotics_backups.log 2>&1
# ===========
# =========== Backup robotics.mvla.net:/var/www
# ===========     Run it at 3:50 am.
    43      20     *          *         *       (echo -e "\n============ "; date ; rsync -av  --log-file=$HOME/log/rsync_robotics.log   michael@robotics.mvla.net:/var/www /backups/SpartanRobotics/backups_from_robotics.mvla.net_via_rsync ) >> $HOME/log/rsync_stdo_robotics_www.log 2>&1

Security

2015-07-05  Brian told me about the fail2ban package that locks out attackers that cause too many log messages.  I installed it and it immediately disabled an IP address. apt-get install ntp

Basic Configuration

2015-05-27 I installed the NTP daemon to keep the time correctly at Brian's suggestion.  apt-get install ntp

Spartan Robotics Drupal Website:

2014-05-01 I created some detailed installation notes when I moved the SpartanRobotics.org website from Bruce Moxin's server to robotics.mval.net

Apache

Apache documentation.  We are running Apache version 2.2 on robotics.mvla.net.

  • Directives in the Configuration files may apply to the entire server, or they may be restricted to apply only to particular directories, files, hosts, or URLs. This document describes how to use configuration section containers or .htaccess files to change the scope of other configuration directives.
  • Index to directives.
  • I use the server to host the Los Altos Robotics website.  Here are my notes for how I set up Apache to point to the Plone server.
  • On October 12, 2014, Douglas and Michael had trouble downloading a robot log file that had been compressed.  The file had a ".txt.gz" extension on it.  We figured out that it was being compressed by Apache and not decompressed by the browser.  This double compression resulted in the file appearing to be a mess.  This bug report talks about it and this How to exclude urls from Apache mod_deflate via SetEnvIfNoCase post offers a solution.  It looks like Apache is seeing the ".txt" part of the extension and compressing the file as if it is a text file and that Apache is ignoring the ".gz" part.  We were able to download files with out the ".txt" ending in ".gz" and have them be the same on our client as they were on the server.  The compression is handled by mod_deflate.  The Apache version 2.2 mod_deflate manual has some different and informative examples in it so it might be worth checking out too.  Below is the mod_deflate configuration file that worked.  I restarted the server with "service apache2 reload".  Brian helped out by writing the "no-gzip" line and locating the solution post.  Michael suspects that the "MIME-TYPE" for text files is poorly written and does not see the ".gz" closing enxtension.  It might work fine if a "$" is added to it.  Michael Oct 13, 2014
     <IfModule mod_deflate.c>
    # these are known to be safe with MSIE 6
    AddOutputFilterByType DEFLATE text/html text/plain text/xml
    
    # everything else may cause problems with MSIE 6
    AddOutputFilterByType DEFLATE text/css
    AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript
    AddOutputFilterByType DEFLATE application/rss+xml
    
    # Brian suggested adding this line so that already compressed files are not compressed a second time.
    # I took the answer from
    # http://stackoverflow.com/questions/11440448/how-to-exclude-urls-from-apache-mod-deflate-via-setenvifnocase
    # and tweaked it.
    # I read the mod_deflate man page http://httpd.apache.org/docs/2.2/mod/mod_deflate.html
    # and it had some good information in it too.
    # Oct 13, 2014. Michael
    SetEnvIfNoCase Request_URI \.(?:gz|bz2|xz)$ no-gzip
    </IfModule>
    

Network Configuration:

The machine has one network interface.  The eth0 network is connected to the school network with 172.20.252.61 default route 172.20.252.254 and netmask of 255.255.0.0.  External to the school, the name is robotics.mvla.net with address: 205.167.46.61.  The server was moved to the 172.20.252.61 address on Feb 7, 2014 by Michael and Jack.  Jack set it up so we can ping the machine, send email to it, ssh to it, and use http and https.  Jack said that ports 1-32 are for the student network, 33-38 are for the teacher network, and 39-48 are for the wireless.

root@robotics:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:01:c0:08:b2:95
inet addr:172.20.252.61 Bcast:172.20.255.255 Mask:255.255.0.0
inet6 addr: fe80::201:c0ff:fe08:b295/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:329739 errors:0 dropped:1 overruns:0 frame:0
TX packets:128574 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:33126632 (31.5 MiB) TX bytes:134835716 (128.5 MiB)
Interrupt:40 

root@robotics:~# ip route
default via 172.20.252.254 dev eth0
172.20.0.0/16 dev eth0 proto kernel scope link src 172.20.252.61

root@robotics:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 172.20.252.254 0.0.0.0 UG 0 0 0 eth0
172.20.0.0 * 255.255.0.0 U 0 0 0 eth0

root@robotics:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.20.252.254 0.0.0.0 UG 0 0 0 eth0
172.20.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0

root@robotics:~# cat /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # DHCP configuration has these two lines. #auto eth0 #iface eth0 inet dhcp # Fixed IP configuration has these lines. # Enter the nameserve address in /etc/resolv.conf by hand after switching from # DHCP to manually configured. Use 8.8.4.4 and 8.8.8.8 auto eth0 allow-hotplug eth0 iface eth0 inet static address 172.20.252.61 netmask 255.255.0.0 network 172.20.252.0 gateway 172.20.252.254 dns-nameservers 8.8.4.4

Notes from Jan 2014 installation on fit-PC

Brian and Michael installed Linux on it.  The first section of these notes covers the Linux installation.  The next section talks about how the machine was configured to work as our robotics.mvla.net SVN server.  Unfortunately, I ran out of energy to document all of the steps required so some of the last steps are not written down.  They involved making links and setting up the backups.  Details on the fit-PC2 hardware are located at the top of this page.  Michael Schuh, Jan 9, 2014.

  • Hit F2 while booting to get to the Prom BIOS configuration.  Make sure the internal hard disk boot device is the lowest in the list.
  • Set up a "Flash drive" (USB Thumb drive) following the directions at http://www.debian.org/releases/stable/i386/ch04s03.html.en .  Try to find one that does not try to install drivers when inserted into a Windows computer.
  • i386 on CD netinst version.  http://www.debian.org/CD/http-ftp/#stable
  • Insert in USB port and connect power.  This should bring up the Install selector and select "Install".
  • Select software sets
    • Web server
    • Mail server
    • SSH server
    • Standard system utilities
  • After booting it, install firmware with dpk
  • dpkg -i firmware-realtek_*
  • dhclient eth0 # Starts up the network by hand
  • apt-get update
  • apt-get install vim
  • vi /etc/apt/sources-list and add "contrib non-free" to each line.
  • apt-get update
  • apt-get install firmware-linux-nonfree rsync xz-utils less
  • shutdown -r now
  • dhclient eth0 # Starts up the network by hand
  • Use "dmesg" to show the kernel log message.
  • apt-get install subversion subversion-tools git rcs vim-gtk
  • vi /etc/network/interfaces and add "\nauto eth0\niface eth0 inet dhcp" to the end of the file.
  • shutdown -r now  # Checking 
  • https://help.ubuntu.com/community/AutomaticSecurityUpdates has information on setting up automatic installation of security patches.
  • apt-get install websvn # reconfigure with "dpkg-reconfigure websvn"
  • copy /etc/{shadow,passwd,group} to the new machine and copy user account entries into it.
  • Use tar to copy accounts in /home to the new machine.
  • Set the date: date 121920462013
  • Copy svn repository backups over and restore them.
    • mkdir /var/svn-repos
    • Copy backups to /var/svn-repos/tmp
    • On helium in /backups/SpartanRobotics/rsync/backups/svn/robotics.mvla.net.daily run
      scp -p frc971-training2013-22.tar.bz2 frc971-4622.tar.bz2 frc971_4623-4626.WeeklyIncr.dump 192.168.0.30:/var/svn-repos/tmp
    • cd /var/svn-repos
    • tar -xvf tmp/frc971-4622.tar.bz2 
  • Working on setting up WebSVN - Refer to http://www.howtoforge.com/debian_subversion_websvn
    • cd /var/svn-repos
    • chown -R www-data.subversion frc971
  • Install NPT daemon to keep the time correctly on the machine - Brian suggested this May 13, 2015
    • apt-get install ntp

I found my old notes on how I built the helium server at the house.  I followed those directions.  I updated them some, but not completely.  They can be used as a guide if needed.

  • Looked at differences in the package list and installed
    • apt-get install rcs ruby csh tcsh ed bzip2 zip ispell  make samba vim unzip subversion samba-doc samba-doc-pdf lynx sudo bind9 g++ gcc libssl-dev build-essential 
  • cd /tmp
  • ssh robotics dpkg -l > s
  • dpkg -l > s2
  • awk '{print $2}' s > s.1
  • 517  awk '{print $2}' s2 > s2.1
  • diff s.1 s2.1
  • apt-get install tcsh sudo ruby irb ed libapache2-svn
  • Skipped 
  • Copy issue.net file for ssh logins: cd /etc; mkdir RCS; ci -l issue.net; scp helium:/etc/issue.net/etc/issue.net
  • Setting up svn server
    • Setting up Apache and Subversion on Debian Squeeze - follow these directions
    • apt-get install websvn subversion-tools subversion libapache2-svn
      • Tell it to use "/var/svn-repos" rather than "/usr/var/svn"
      • Note on permissions from the installer:                                                                                                                                                 
           Due to a limitation in the DB format, the 'svnlook' command needs read-write access to the repository (to create locks etc). You need to give         
           read-write permissions to the user running your webserver on all your repositories.                                                                   
                                                                                                                                                                 
           Another way of avoiding this problem is by creating SVN repositories with the --fs-type=fsfs option.  Existing DB repositories can be converted to    
           the FSFS format by using the svnadmin dump/load commands.
    • mkdir /var/svn-repos ; scp -rp tin:/opt/subversion/etc /var/svn-repos (copy the repositories over or preferably restore them from backups) ; chown -R www-data.www-data /var/svn-repos/etc
    • Make sure paths are correct in /etc/websvn.  Look at the old server to see how this is done.  No need to restart apache2 after doing this.
      • Run "dpkg-reconfigure websvn" to make sure that the path is set to /var/svn-repos.  I left the "apache, apache-ssl, apache-perl, and apache2" options all checked.  I answered "/var/svn-repos" to both the "location of each parent directory" and "location of each svn repository" questions.  This updates the /etc/websvn/svn_deb_conf.inc file.
      • Set the path in the config.phg file to "$config->useAuthenticationFile('/var/svn-repos/authz'); // Global access file"
      • This page talks about setting up Web_SVN and the /etc/websvn config files.  http://www.howtoforge.com/debian_subversion_websvn
    • cd /etc/apache2/mods-available ; mkdir RCS; ci -l dav_svn.conf; Add/uncomment the following lines (or better yet, copy the file from the old server):
      <Location /svn>
         DAV svn
         SVNParentPath /var/svn-repos
         AuthType Basic
         AuthName "Subversion Repository"
         AuthUserFile /etc/apache2/dav_svn.passwd
         AuthzSVNAccessFile /var/svn-repos/authz
         Require valid-user
         SSLRequireSSL
      </Location>
    • /etc/apache2 (copy the dav_svn.passwd file from the old server)
         dav_svn.passwd - password file.  Use htpasswd to add or change a password
                          htpasswd -s /etc/apache2/dav_svn.passwd austin    # Add -c flag to create the file.
                          htpasswd -b -s /etc/apache2/dav_svn.passwd austin password # Batch mode with password on the command line.
    • enable ssl, dav, and dav_svn with
      #      a2enmod ssl 
           root@robotics2:/etc/apache2# a2enmod ssl
           Enabling module ssl.
           See /usr/share/doc/apache2.2-common/README.Debian.gz on how to configure SSL and create self-signed certificates.
           To activate the new configuration, you need to run:
                 service apache2 restart
    • #      a2enmod dav
      #      a2enmod dav_svn
    • Create a /var/svn-repos/authz file (copy from old server) with
      [etc:/]
      michael = rw
    • mkdir /etc/apache2/ssl ; cd /etc/apache2/ssl ; openssl req -new -x509 -days 400 -nodes -out apache.pem -keyout apache.key  # Use Michael.SchuhFamilySSL@Boardsailor.com for email.
    • Went through directions at http://www.howtoforge.com/debian_subversion_websvn
        501  cd /etc/apache2/sites-available/
        511  scp michael@robotics.mvla.net:/etc/apache2/sites-enabled/robotics schuhSVN (Copy robotics from old server instead of schuhSVN)
        513  vi schuhSVN 
      # cat schuhSVN
      # Enable this site by running: a2ensite schuhSVN
      # and then restart apache2: /etc/init.d/apache2 restart
      # Also need to enable ssl, dav, and dav_svn with
      # a2enmod ssl
      # a2enmod dav
      # a2enmod dav_svn
      #
      # Also see http://blog.mattsch.com/2011/03/21/setting-up-apache-and-subversion-on-debian-squeeze/
      # and http://www.howtoforge.com/debian_subversion_websvn
      
      
      ServerAdmin webmaster@example.com
      ServerName svn.example.com
      DocumentRoot /usr/share/websvn/
      
      Options FollowSymLinks
      order allow,deny
      allow from all
      AuthType Basic
      AuthName "Subversion Repository"
      Require valid-user
      AuthUserFile /etc/apache2/dav_svn.passwd
      
      php_flag magic_quotes_gpc Off
      php_flag track_vars On
      
      
      SSLEngine on
      SSLCertificateFile /etc/apache2/ssl/apache.pem
      SSLCertificateKeyFile /etc/apache2/ssl/apache.key
      
      
        514  cd ..
        519  a2ensite schuhSVN (a2ensite robotics)
               
        a2enmod authz_svn
    • 520  /etc/init.d/apache2 restart (service apache2 restart) is the new way.
      527  chown www-data.www-data /var/svn-repos//au*
    •   530  chmod go+r /var/svn-repos//au*  # I bet that this is not needed.
        528  l /var/svn-repos/
        529  cd /tmp; svn co https://losaltosrobotics/svn/etc   # Use this for testing.
        531  svn co https://losaltosrobotics/svn/etc
      Browse to https://robotics.mvla.net/ or https://192.168.0.10/
    • svnadmin: Can't open file '/pathtorepo/db/fsfs.conf': No such file or directory" Error message from svn-hot-backup script
      • This page says the problem is that SVN 1.6 changed things.  It suggests doing an "svnadmin upgrade /path/toRepository" and adding the offending file as one option.  The other is to do a dump, create a new repository, and then do an "svn load".  He claimed that this would make the most optimal copy of the repository.  I tried this with etc.  I am not sure it was worth the optimal part of the claim.
        • svnadmin dump etc > etc.dump
        • mv etc etc.old
        • svnadmin create etc
        • svnadmin load etc < etc.dump   # This was slow on the plug computer.
    • Make files and git links in /usr/share/websvn
    • cd /usr/share/websvn
    • ln -s  /www/https/files/ files
    • ln -s  /www/https/git git
  • Set up links in /var/www
    • cd /var/www
    • ln -s /www/http/files files
    • ln -s /www/http/tmp tmp
    • cp (from old machine):/var/www/index.html /var/www/index.html # Has "Hi Travis" in it.
  • Set up links in /root/bin
    • root@robotics2:~/bin# l 
    • lrwxrwxrwx 1 root root 55 Jan  1 17:50 backupOnlyChangedArchives.csh -> ../local/etc/bin_robotics/backupOnlyChangedArchives.csh
    • lrwxrwxrwx 1 root root 60 Jan  1 17:51 backupOnlyChangedArchivesDaily.csh -> ../local/etc/bin_robotics/backupOnlyChangedArchivesDaily.csh
    • lrwxrwxrwx 1 root root 61 Jan  1 17:51 backupOnlyChangedArchivesWeekly.csh -> ../local/etc/bin_robotics/backupOnlyChangedArchivesWeekly.csh
    • lrwxrwxrwx 1 root root 36 Jan  1 17:51 backupSchuhComputers.rb -> ../local/etc/backupSchuhComputers.rb
    • lrwxrwxrwx 1 root root 39 Jan  1 17:51 cronfile.root -> ../local/etc/bin_robotics/cronfile.root
    • lrwxrwxrwx 1 root root 47 Jan  1 17:52 make_index -> /www/https/files/frc971/2013/michael/make_index
  • Set up disk mounts
    • mkdir /seagate /backups /backupsSecondary /backupsNoBackup
    • cat >> /etc/fstab   # The "nofail" makes it so the server boots fine without the external USB disk attached.  Otherwise it hangs.
              UUID=9cd8be3e-b2ca-40dc-9336-ed53d7b67547 /seagate         ext3    rw,nofail 0       1
              /seagate/backups                      /backups                               none    rw,bind                 0 0 
              /seagate/backupsSecondary       /backupsSecondary               none    rw,bind                 0 0
              /seagate/backupsNoBackup        /backupsNoBackup               none    rw,bind                 0 0
              /seagate/www                            /www                                    none    rw,bind                 0 0
  • Set up Backups - 
    • mkdir -p /var/schuh/log /var/schuh/backups
      • mkdir -p /var/schuh/log /var/schuh/backups
      • ln -s /var/svn-repos/backup_robotics.log /var/schuh/log
      • ln -s /var/svn-repos/svn_cron_backups.log /var/schuh/log
      • cat "var/svn-repos >> /etc/backupSchuhComputers.exclude
    • ln -s ../schuh /var/log/schuh # I could not figure out what this one was for so I did not do it.  1/1/14.
    • If needed, compile a version of gnutar that will do the incremental backups and put the tar file in ~root/bin
  • Use cron jobs to backup svn and plone to /backups.  Backup log messages are written to /var/schuh/log.
    # List the root cronfile with: crontab -l
    # Submit this cronfile with: crontab cronfile.root
    #
    # ~/bin/cronfile.root
    #
    # Michael Schuh
    # August 14, 2010
    
    SHELL=/bin/bash
    
    # =============== Backup the helium root filesystem
    # Make a full backup once a quarter on the last day of the month. Note: only run if tomorrow is the first day of the month.
    30 22 * 3,6,9,12 * [ `date -d tomorrow +%d` -eq '01' ] && $HOME/bin/backupSchuhComputers.rb -0 >> /var/schuh/log/backup_helium.log 2>&1
    # Do a weekly backup every Sunday morning at just before midnight
    30 22 * * Sun $HOME/bin/backupSchuhComputers.rb -1 >> /var/schuh/log/backup_helium.log 2>&1
    # Do a daily backup every day except Sunday just before midnight
    30 22 * * 0-6 $HOME/bin/backupSchuhComputers.rb -2 >> /var/schuh/log/backup_helium.log 2>&1
    
    
    # =============== Backup the Los Altos Robotics Plone site every night. The script makes a copy in with the Plone website instance
    # in the /backups/linux/$hostname/plone directory. Logs are in /var/schuh/log
    #
    # min hour dayOfMonth month dayOfWeek command
    #55 18 * * * date >> /tmp/crontab.date
    40 13 * * * $HOME/bin/backup_LosAltosRoboticsWebsitePloneDatabase.plone4 >> /var/schuh/log/backup_LosAltosRoboticsWebsitePloneDatabasePlone
    4.log 2>&1
    40 23 * * * $HOME/bin/backup_LosAltosRoboticsWebsitePloneDatabase.plone4 >> /var/schuh/log/backup_LosAltosRoboticsWebsitePloneDatabasePlone
    4.log 2>&1
    # Backup the blobstorage files. They are needed in addition to the Data.fs file to recover a plone website.
    # Make a full backup once a quarter on the last day of the month. Note: only run if tomorrow is the first day of the month.
    55 23 * 3,6,9,12 * [ `date -d tomorrow +%d` -eq '01' ] && $HOME/bin/backupPloneBlobstorageFiles.rb -0 >> /var/schuh/log/backup_LosAltosRoboticsWeb
    sitePloneDatabasePlone4_blobstorage.log 2>&1
    # Do a weekly backup every Sunday morning at just before midnight
    55 23 * * Sun $HOME/bin/backupPloneBlobstorageFiles.rb -1 >> /var/schuh/log/backup_LosAltosRoboticsWebsitePloneDatabasePlone4_blobstorage.log
    2>&1
    # Do a daily backup every day except Sunday just before midnight
    55 23 * * 0-6 $HOME/bin/backupPloneBlobstorageFiles.rb -2 >> /var/schuh/log/backup_LosAltosRoboticsWebsitePloneDatabasePlone4_blobstorage.log
    2>&1
    
    
    # =============== Backup SVN repositories to /backups on the external disk.
    # SVN Backup logs are in /var/schuh/log
    # Make a full back of repositories once a quarter on the last day of the month. Only repositories that have been changed are backed up.
    31 1 * 3,6,9,12 * [ `date -d tomorrow +%d` -eq '01' ] && $HOME/bin/backupOnlyChangedArchives.csh >> /var/schuh/log/svn_cron_backups.log 2>&1
    # Do a weekly backup every Sunday morning at 4 am
    0 4 * * Sun $HOME/bin/backupOnlyChangedArchivesWeekly.csh >> /var/schuh/log/svn_cron_backups.log 2>&1
    # Do a daily backup every morning at 4 am
    0 4 * * * $HOME/bin/backupOnlyChangedArchivesDaily.csh >> /var/schuh/log/svn_cron_backups.log 2>&1
  • Set up automounting of backup disk at boot time.
    • Added "mount -a " to /etc/rc.local".  This is probably how I should add the plone startup.
    • Added the line "/dev/sda1 /backups ext3 rw 0 0" to the /etc/fstab file.
  • Copy ssh keys from helium:/etc/ssh to new machine:
    -rw-------  1 root root    668 Feb  7  2010 ssh_host_dsa_key
    -rw-r--r--  1 root root    601 Feb  7  2010 ssh_host_dsa_key.pub
    -rw-------  1 root root   1671 Feb  7  2010 ssh_host_rsa_key
    -rw-r--r--  1 root root    393 Feb  7  2010 ssh_host_rsa_key.pub
  • Clean up dpkg package download files from /var/cache/apt with: apt-get clean
Final steps to do before going live.
  • Take the robotics server off line
  • Get a current copy of the repositories, /etc/apache2/dav_svn.passwd, /var/svn-repos/{authz,svn_cron_backups.log,RCS}
  • Add /usr/share/websvn to the svn tools backup script. (/media/usb0/svn-local-bin/backupScripts.csh )
  • Set up a cron job or some other way of backing up the svn tools and the svn-local-bin directory.

Notes from Dec 2011 installation:

robotics:/media/usb0/svn-repos# cat ~michael/notes.txt 
/home/michael/notes.txt
These notes

Directories:

/etc/apache2/sites-available
robotics - This sets up ssl to use websvn
Enable the robotics site by running:
a2ensite robotics
apache2ctl restart
Another way to restart apache2 is: /etc/init.d/apache2 restart

/etc/apache2
dav_svn.passwd - password file. Use htpasswd to add or change a password
htpasswd -s /etc/apache2/dav_svn.passwd austin
htpasswd -b -s /etc/apache2/dav_svn.passwd austin password # Batch mode with password on the command line.

/etc/apache2/ssl
has apache.{key,pem} files created with "openssl req -new -x509 -days 400 -nodes -out apache.pem -keyout apache.key"
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:California
Locality Name (eg, city) []:Mountain View
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Mountain View High School
Organizational Unit Name (eg, section) []:Spartan Robotics FRC Team 971
Common Name (eg, YOUR name) []:robotics.mvla.net
Email Address []:michael.FRC971svnCertificate@bo.......or.com
     Restart the server with: /etc/init.d/apache2 restart
/etc/apache2/mods-available
dav_svn.conf - makes it so that "svn co https://robotics.mvla.net/svn/frc971"
goes to the correct place. Also sets svn up to require a
valid user to access the files.
- The values set in dav_svn.conf are:
DAV svn
SVNParentPath /var/svn-repos
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
AuthzSVNAccessFile /var/svn-repos/authz # This file controls the access to the repository
Require valid-user
SSLRequireSSL

/etc/websvn
svn_deb_conf.inc - tells websvn where the repository is. Use
"dpkg-reconfigure websvn" to change things.

/var/svn-repos
This is where the repositories live.

/var/svn-repos/authz # This file controls the access to the repository
# I figured the path out by reading http://svnbook.red-bean.com/en/1.1/svn-book.html#svn-ch-6-sect-4.4.2
[frc:/2011] # This and the next line allow user "michael" access to the 2011 directory in the "frc" repository.
michael = rw
File permissions matter. This works:
-rwxr-x--- 1 www-data subversion 1250 2011-12-05 08:17 authz

/var/log/apache2
Log files.

# Also need to enable ssl, dav, and dav_svn with
# a2enmod ssl
# a2enmod dav
# a2enmod dav_svn
#
# Also see http://blog.mattsch.com/2011/03/21/setting-up-apache-and-subversion-on-debian-squeeze/
# and http://www.howtoforge.com/debian_subversion_websvn
# in the "howtoforge.com" instructions, skip the enscript part
# as it is no longer needed.


Installed packages include:
apt-get install subversion-tools

Change the system timezone with
dpkg-reconfigure tzdata

To update the timezone, use
ntpdate time.nist.gov


I think apt-get was pulling packages from the squeeze distribution for a while before Austin changed
the sources.list file to point to lenny packages. It appears from the date on the sources.list file
this was changed July 2011. Here are some packages that look wrong.

michael@robotics:~$ dpkg -l | grep squee
ii base-files 6.0squeeze2 Debian base system miscellaneous files
ii dbus-x11 1.2.24-4+squeeze1 simple interprocess messaging system (X11 deps)
ii dhcp3-client 4.1.1-P1-15+squeeze2 ISC DHCP server (transitional package)
ii isc-dhcp-client 4.1.1-P1-15+squeeze2 ISC DHCP client
ii isc-dhcp-common 4.1.1-P1-15+squeeze2 common files used by all the isc-dhcp* packages
ii libdbus-1-3 1.2.24-4+squeeze1 simple interprocess messaging system
ii libgssapi-krb5-2 1.8.3+dfsg-4squeeze1 MIT Kerberos runtime libraries - krb5 GSS-API Mechanism
ii libk5crypto3 1.8.3+dfsg-4squeeze1 MIT Kerberos runtime libraries - Crypto Library
ii libkrb5-3 1.8.3+dfsg-4squeeze1 MIT Kerberos runtime libraries
ii libkrb53 1.8.3+dfsg-4squeeze1 transitional package for MIT Kerberos libraries
ii libkrb5support0 1.8.3+dfsg-4squeeze1 MIT Kerberos runtime libraries - Support library
ii libssl0.9.8 0.9.8o-4squeeze1 SSL shared libraries
ii libxml2 2.7.8.dfsg-2+squeeze1 GNOME XML library
ii lsb-base 3.2-23.2squeeze1 Linux Standard Base 3.2 init script functionality
ii openssl 0.9.8o-4squeeze1 Secure Socket Layer (SSL) binary and related cryptographic tools
ii passwd 1:4.1.4.2+svn3283-2+squeeze1 change and administer password and group data
ii tzdata 2011d-0squeeze1 time zone and daylight-saving time data

Austin took a shot at fixing all of this on Nov 28, 2011. Here are his notes.
Austin: Ok, I think I have it all working again...
Sent at 7:51 PM on Monday
Austin: I don't really have much in the way of notes, other than that the package versions were off. I fixed this by using dpkg -l | grep squeeze to find the bad packages, and then apt-get install packagename/oldstable to downgrade it. I then fought long and hard.
I'm going to go to bed, but if you would like me to detail a bit more what I did, I can.
Sent at 7:54 PM on Monday
me: Thanks. Your notes are good enough. No need to spend more time on them.
Thank you for sorting things out on robotics.mvla.net. I appreciate it.
Sent at 9:07 PM on Monday

# /media/usb0/svn-local-bin/README Describes the scripts
# Use the backupScripts.csh to backup all the scripts and configuration files that
# Michael has changed and remembered to include in the script.
#
ls -l /media/usb0/svn-local-bin
-rwxr-xr-x 1 root root 1331 2011-12-03 15:54 backupNowToLocalDisk.csh
-rwxr-xr-x 1 root root 4358 2011-12-03 16:59 backupOnlyChangedArchives.csh
-rwxr-xr-x 1 root root 4674 2011-12-03 17:40 backupOnlyChangedArchivesDaily.csh
-rwxr-xr-x 1 root root 4063 2011-12-03 17:51 backupOnlyChangedArchivesWeekly.csh
-rwxr-xr-x 1 root root 1219 2011-12-03 21:15 backupScripts.csh
-rwxr-xr-x 1 root root 21 2011-11-26 20:18 csh_check
drwxr-xr-x 2 root root 4096 2011-12-03 20:48 RCS
-rw-r--r-- 1 root root 593 2011-12-03 21:16 README

/home/michael/backupSchuhComputers.rb
Use this to backup the machine.


#=======================================
#
# The USB drive was being mounted with noexec at boot time. This prevents
# us from running scripts from it. This does not work well. The fstab
# entry was also being ignored. Austin suggested removing "hal". I did so
# on 12/26/11. Michael
#
# from "fstab -l"
# /dev/sdc1 on /media/usb0 type ext3 (rw,noexec,nodev,sync,noatime)
#
#
robotics:/# dpkg -l | grep hal
ii hal 0.5.11-8 Hardware Abstraction Layer
ii hal-info 20091130-1 Hardware Abstraction Layer - fdi files
ii libhal-storage1 0.5.11-8 Hardware Abstraction Layer - shared library
ii libhal1 0.5.14-3 Hardware Abstraction Layer - shared library
robotics:/# dpkg --purge hal hal-info libhal-storage1 libhal1


# It turns out that this did not fix the problem.
#
# The USB hard disk kept showing up with "noexec" at boot time. It took
# a while to figure out that usbmount was being called by udev and mounting it.
# We would much rather mount it by using the fstab entry than udev/usbmount.
#
# To turn of usbmount, I moved /etc/udev/rules.d/z60_usbmount.rules to noLongerUsed.rules.d

robotics:/etc/udev# l noLongerUsed.rules.d/
total 0
lrwxrwxrwx 1 root root 17 2010-04-06 03:55 z60_usbmount.rules -> ../usbmount.rules

# This fixed the mounting incorrectly problem. Had to leave the "mount -a" in /etc/rc.local
# script.
#
# Michael

System Log

11/21/2015 Set up CAD machines to use different route to internet

 The power to the MVHS campus was turned off 11/21/15 until 7 am.  When it came back up, the network would not let machines on the student network out of the campus.  Jack told me how to configure the network on the machines on the student network so that they would work on the internet.

To get out of the MVHS Robotics classroom when the student network does not work,

On a machine on the student network, use

IP 172.20.21.x (Get this by seeing what the machine was assigned by the switch)
Subnet mask: 255.255.0.0
Default gateway 172.20.252.254
DNS Server 172.20.253.6 (He also said that we could use 8.8.8.8 - Google)

Robotics.mvla.net has an internal address of 172.20.252.61 and the external address is 205.167.46.61

10/1/2015 IP Address for printer in Ly's room

 Todd gave me an IP address for the printer in Ly's room.  He told me: "Yes, I have been in touch with Jack.  You can try the address 172.20.196.8 by pinging it to see if it’s free.  If not just go up by one digit to see if the next one is available.  Let me know if you have problems.  I have about 4 more of those printer available if you need extras."

7/5/2015 Added fail2ban

5/27/2015 Added NTP daemon to keep the time correct

2/7/2014  Changed the IP address of the machine

Jack requested that we change the IP address.  So now the eth0 network is connected to the school network with 172.20.252.61 default route 172.20.252.254 and netmask of 255.255.0.0.  External to the school, the name is robotics.mvla.net with address: 205.167.46.61.  Jack set it up so we can ping the machine, send email to it, ssh to it, and use http and https.  Michael

1/12/2014   Setup gitweb for Brian

Dave Smith and Michael set up gitweb so the 971 svn users can view the 971 git repositories.  While we were at it, we used an environment variable to specify team specific /etc/gitweb{114,971}.conf files for the teams in /etc/apache2/sites-available/robotics.

  • apt-get install gitweb
  • Added customizations to /etc/apache2/sites-available/robotics
  • Made custom /etc/gitweb{114,971}.conf files from /etc/gitweb.conf.  Had them point to "/var/cache/git{114,971}".
  • Link in user git repositories into /var/cache/git{114,971}
    • mkdir /var/cache/git971
    • cd /var/cache/git971
    • ln -s /www/https/git/frc971/brian/2013.git brian_2013.git  # do for each git repository.
  • Replaced symbolic links in the user git repositories for object to have absolute paths
    • ln -s /www/https/git/frc971/objects/2013 /www/https/git/frc971/brian/2013.git/objects
  • Wrote /etc/apache2/make_git.passwd to create /etc/apache2/git{114,971}.passwd files using the team groups in /var/svn-repos/authz and the passwords from /etc/apache2/dav_svn.passwd.

4/13/2012  Does not connect to the network after MVHS power failure.

The server does not connect to the network after a power failure.  "ping goole.com" fails.  It appears that the routing table is messed up.  "ip route" does not show the 192.168.10.1 default route.  After reading this page, I ran "# route add default gw 192.168.10.1 eth0" and "ping google.com" and everything else worked. 

I rebooted the machine with the network cable unplugged, plugged it in, and could not ping Google.  I did an "ifconfig eth0 down" and then an "ifconfig eth0 up" and "ip route" came back with the 192.168.10.1 default route missing.  I ran the "route add default gw 192.168.10.1 eth0" command again and everything worked again.

robotics:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     *               255.255.255.0   U     0      0        0 eth1
192.168.10.0    *               255.255.255.0   U     0      0        0 eth0
default         192.168.10.1    0.0.0.0         UG    0      0        0 eth0

1/7/2012  SVN repository error

frc971_bad_20120107_michael had a problem with a file in revision 2603 2006/trunk/pictures/2006_03_17. When doing an update on this, an error message was generated and the apache2 error.log file said there was a checksum problem.

svnadmin verify /media/usb0/svn-repos/frc971 # This said there was a Checksum mismatch on version 2603.
cd /media/usb0/backups/svn/robotics.mvla.net.daily
tar -xvf frc971-2527.tar.bz2 # This took about an hour. The unzip was painfully slow.
svnadmin load frc971-2527 < frc971_2528-2594.WeeklyIncr.dump
svnadmin load frc971-2527 < frc971_2595-2611.DailyIncr.dump # I checked this version and it worked so I loaded the rest of the updates below.
mv frc971-2527 ../svn-repos/frc971recovered
cd ../svn-repos
cat frc971/db/current # Shows the current revision of the active repository.
svnadmin load frc971recovered < ../backups/svn/robotics.mvla.net.daily/frc971_2612-2614.DailyIncr.dump
svnadmin load frc971recovered < ../backups/svn/robotics.mvla.net.daily/frc971_2615-2617.DailyIncr.dump
svnadmin load frc971recovered < ../backups/svn/robotics.mvla.net.daily/frc971_2618-2621.DailyIncr.dump
svnadmin load frc971recovered < ../backups/svn/robotics.mvla.net.daily/frc971_2622-2625.DailyIncr.dump
svnadmin load frc971recovered < ../backups/svn/robotics.mvla.net.daily/frc971_2626-2626.DailyIncr.dump
diff -r frc971 frc971recovered
diff -r frc971/conf/set_passwords.csh frc971recovered/conf/set_passwords.csh
cp frc971/conf/set_passwords.csh frc971recovered/conf/set_passwords.csh
chown -R www-data.subversion frc971recovered/
mv frc971 frc971_bad_20120107_michael
mv frc971recovered frc971

robotics:/tmp/2006_03_17# svn up
svn: REPORT of '/svn/frc971/!svn/vcc/default': Could not read chunk delimiter: Secure connection truncated (https://robotics.mvla.net)

robotics:/tmp/2006_03_17# svn info
Path: .
URL: https://robotics.mvla.net/svn/frc971/2006/trunk/pictures/2006_03_17
Repository Root: https://robotics.mvla.net/svn/frc971
Repository UUID: f308d9b7-e957-4cde-b6ac-9a88185e7312
Revision: 2626
Node Kind: directory
Schedule: normal

robotics:/var/log/apache2# tail error.log
[Sat Jan 07 19:47:22 2012] [error] [client 192.168.10.61] Provider encountered an error while streaming a REPORT response. [500, #0]
[Sat Jan 07 19:47:22 2012] [error] [client 192.168.10.61] A failure occurred while driving the update report editor [500, #160004]
[Sat Jan 07 19:47:22 2012] [error] [client 192.168.10.61] Checksum mismatch while reading representation:\n expected: 54ded6bf34052dfe01cd67b50229d175\n actual:
f2d117f27f971408f37b3c6bb028ea33\n [500, #160004]

12/8/2011 - Fixed network problems.

Jack,

Thank you very much for the help today.  I told my 22 year old son, Austin, what you found with me on the server and he agreed that the second gateway was wrong.  He also noticed that I had it configured to use the nameserver at our house in addition to the MVHS nameserver and this was causing "ping google.com" to be slow.  He fixed both of these.  He updated the /etc/network/interfaces file and rebooting to make sure the machine will come up with a proper configuration.  Here is what we have now:

# Used by ifup(8) and ifdown(8). See the interfaces(5) manpage or
# /usr/share/doc/ifupdown/examples for more information.
auto lo
iface lo inet loopback

# eth0 is the top ethernet plug on the server
# eth1 is the bottom ethernet plug on the server

# 192.168.10.61 is the MVLA internal IP address for robotics.mvla.net
auto eth0
iface eth0 inet static
     address 192.168.10.61
     network 192.168.10.0
     netmask 255.255.255.0
     gateway 192.168.10.1
     dns-nameservers 205.167.47.141

auto eth1
iface eth1 inet static
     address 192.168.1.67
     network 192.168.1.0
     netmask 255.255.255.0

Here are three different ways of looking at the routing for the server:

robotics:~# ip route
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.67
192.168.10.0/24 dev eth0  proto kernel  scope link  src 192.168.10.61
default via 192.168.10.1 dev eth0

robotics:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     *               255.255.255.0   U     0      0        0 eth1
192.168.10.0    *               255.255.255.0   U     0      0        0 eth0
default         192.168.10.1    0.0.0.0         UG    0      0        0 eth0

robotics:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         192.168.10.1    0.0.0.0         UG    0      0        0 eth0

It sure looks to me like the machine is working properly.  It is much more responsive to connection requests.

Thank you again for your expert assistance.  I really appreciate it.

Regards,
   Michael 

 

End of System Log