Appending a script after a running script.

This is a pretty cool function.

I use Gaussian all the time. Sometimes, I just run the command in the terminal at the frontground. But sometimes, I have another job that I want to run after the previous job. I don’t want to check the running condition all the time, and I don’t want to waste the computing resource. So the following is what we can do for this condition.

Got help from HPC manager David Chaffin and Pawel Wolinski, as well as the website at http://www.thegeekstuff.com/2010/05/unix-background-job.

If you have a script 1.sh running in the terminal at the frontground.

Press ctrl+z

Type bg

Then the job will run at the background.

Use jobs to check the scripts running at the background. If 1.sh is the first one, you may type the following command to append your 2.sh. 2.sh will run right after the end of 1.sh.

wait %1; ./2.sh

You may kill jobs by kill %1 too.

The following is the content at http://www.thegeekstuff.com/2010/05/unix-background-job. I think it is very useful.

######################################

Bg, Fg, &, Ctrl-Z – 5 Examples to Manage Unix Background Jobs

When you execute a unix shell-script or command that takes a long time, you can run it as a background job.

In this article, let us review how to execute a job in the background, bring a job to the foreground, view all background jobs, and kill a background job.

1. Executing a background job

Appending an ampersand ( & ) to the command runs the job in the background.

For example, when you execute a find command that might take a lot time to execute, you can put it in the background as shown below. Following example finds all the files under root file system that changed within the last 24 hours.

# find / -ctime -1 > /tmp/changed-file-list.txt &

2. Sending the current foreground job to the background using CTRL-Z and bg command

You can send an already running foreground job to background as explained below:

  • Press ‘CTRL+Z’ which will suspend the current foreground job.
  • Execute bg to make that command to execute in background.

For example, if you’ve forgot to execute a job in a background, you don’t need to kill the current job and start a new background job. Instead, suspend the current job and put it in the background as shown below.

# find / -ctime -1 > /tmp/changed-file-list.txt

# [CTRL-Z]
[2]+  Stopped                 find / -ctime -1 > /tmp/changed-file-list.txt

# bg

3. View all the background jobs using jobs command

You can list out the background jobs with the command jobs. Sample output of jobs command is

# jobs
[1]   Running                 bash download-file.sh &
[2]-  Running                 evolution &
[3]+  Done                    nautilus .

4. Taking a job from the background to the foreground using fg command

You can bring a background job to the foreground using fg command. When executed without arguments, it will take the most recent background job to the foreground.

# fg

If you have multiple background ground jobs, and would want to bring a certain job to the foreground, execute jobs command which will show the job id and command.

In the following example, fg %1 will bring the job#1 (i.e download-file.sh) to the foreground.

# jobs
[1]   Running                 bash download-file.sh &
[2]-  Running                 evolution &
[3]+  Done                    nautilus .

# fg %1

5. Kill a specific background job using kill %

If you want to kill a specific background job use, kill %job-number. For example, to kill the job 2 use

# kill %2

Compare the difference of two Images.

Cited from https://rosettacode.org/wiki/Percentage_difference_between_images#Python.

 

I chose PYTHON script. But the script has some problem. I need to change “import Image” to “From PIL import Image”.

 

The output will give the pixel difference in percentage.

 

But I don’t understand the output. A more clear way is got from http://stackoverflow.com/questions/29229535/measure-similarity-of-two-images-in-java-or-imagemagick. In this method, the different pixel number will be output.

This used ImageMagick in Linux. The command to compare a.png and b.png is:

compare -metric ae a.png b.png null:

The output is the number of different pixel. If you want the percentage. You need to use the following command to get the X and Y for the image and use X*Y to calculate the total pixel. Then use the above result divided by the below result to get the percentage.

identify a.png

Use Jump Host for RSYNC

I have computer A, storage server B and HPCC.

Previously, I use A to SSH B and RSYNC files from HPCC to B. One day, B can not PING or SSH or RSYNC or SCP to HPCC. But B can PING or SSH or RSYNC or SCP to A, and A can also communicate with HPCC. So I am trying to use A as a jump host to help B to download files from HPCC.

 

Got help form https://www.freeture.ch/?p=815.

The first thing is make sure the SSH KEY works between A and B, and between A and HPCC.

In ~/.ssh directory on B, build a new file config. The content is below.

Host remote_server.com
HostName remote_server.com
User remote_server_user_id
ProxyCommand ssh jump_host_user_id@jump_host.com nc remote_server.com 22

6 rsync Examples to Exclude files or folders

Cited from http://www.thegeekstuff.com/2011/01/rsync-exclude-files-and-folders/?utm_source=feedburner

6 rsync Examples to Exclude Multiple Files and Directories using exclude-from

Rsync is very powerful tool to take backups, or sync files and directories between two different locations (or servers).

You know this already, as we presented you with practical examples on rsyncearlier.

In a typical backup situation, you might want to exclude one or more files (or directories) from the backup. You might also want to exclude a specific file type from rsync.

This article explains how to ignore multiple files and/or directories during rsync with examples.

First, create a sample directory structure as shown below (with some empty files) that can be used for testing purpose.

$ cd ~
$ mkdir -p source/dir1/dir2
$ mkdir -p source/dir3
$ touch source/file1.txt
$ touch source/file2.txt
$ touch source/dir1/dir2/file3.txt
$ touch source/dir3/file4.txt

The above command will create a source directory (under your home directory) with the following structure.

source
- file1.txt
- file2.txt
- dir1
  - dir2
    - file3.txt
- dir3
  - file4.txt

1. Exclude a specific directory

If you don’t want to sync the dir1 (including all it’s subdirectories) from the source to the destination folder, use the rsync –exclude option as shown below.

$ rm -rf destination

$ rsync -avz --exclude 'dir1' source/ destination/
building file list ... done
created directory dest
./
file1.txt
file2.txt
dir3/
dir3/file4.txt

Verify to make sure dir1 is not copied from source directory to destination directory.

$ find destination
destination
destination/file2.txt
destination/file1.txt
destination/dir3
destination/dir3/file4.txt

2. Exclude multiple directories that matches a pattern

The following example will exclude any directory (or subdirectories) under source/ that matches the pattern “dir*”

$ rm -rf destination

$ rsync -avz --exclude 'dir*' source/ destination/
building file list ... done
created directory destination
./
file1.txt
file2.txt

Verify the destination directory to make sure it didn’t copy any directories that has the keyword “dir” in it.

$ find destination
destination
destination/file2.txt
destination/file1.txt

3. Exclude a specific file

To exclude a specific file, use the relative path of the file in the exclude option as shown below.

$ rm -rf destination

$ rsync -avz --exclude 'dir1/dir2/file3.txt' source/ destination/
building file list ... done
created directory destination
./
file1.txt
file2.txt
dir1/
dir1/dir2/
dir3/
dir3/file4.txt

Verify the destination directory to make sure it didn’t copy the specific file ( dir1/dir2/file3.txt in this example).

$ find destination
destination
destination/file2.txt
destination/file1.txt
destination/dir1
destination/dir1/dir2
destination/dir3
destination/dir3/file4.txt

4. Exclude path is always relative

If you are not careful, you might make this mistake.

In the following example, the exclude option seems to have a full path (i.e /dir1/dir2/file3.txt). But, from rsync point of view, exclude path is always relative, and it will be treated as dir1/dir2/file3.txt. In the example below, rsync will look for dir1 under source directory (and not under / root directory).

$ rsync -avz --exclude '/dir1/dir2/file3.txt' source/ destination/

So, the above command is exactly same as the following. Just to avoid confusion (and to make it easy to read), don’t give / in front of the exclude path.

$ rsync -avz --exclude 'dir1/dir2/file3.txt' source/ destination/

5. Exclude a specific file type

To exclude a specific file type that has a specific extension, use the appropriate pattern. For example, to exclude all the files that contains .txt as extension, do the following.

$ rsync -avz --exclude '*.txt' source/ destination/
building file list ... done
created directory destination
./
dir1/
dir1/dir2/
dir3/

Verify the destination directory to make sure it didn’t copy the *.txt files.

$ find destination
destination
destination/dir1
destination/dir1/dir2
destination/dir3

Note: The above is very helpful, when you want to backup your home directory, but exclude all those huge image and video files that has a specific file extension.

6. Exclude multiple files and directories at the same time

When you want to exclude multiple files and directories, you can always specify multiple rsync exclude options in the command line as shown below.

$ rsync -avz --exclude file1.txt --exclude dir3/file4.txt source/ destination/

Wait. What if I had tons of files that I want to exclude from rsync?

I can’t keep adding them in the command line using multiple –exclude, which is hard to read, and hard to re-use the rsync command for later.

So, the better way is to use rsync –exclude-from option as shown below, where you can list all the files (and directories) you want to exclude in a file.

First, create a text file with a list of all the files and directories you don’t want to backup. This is the list of files and directories you want to exclude from the rsync.

$ vim exclude-list.txt
file1.txt
dir3/file4.txt

Next, execute the rsync using –exclude-from option with the exclude-list.txt as shown below.

$ rm -rf destination

$ rsync -avz --exclude-from 'exclude-list.txt' source/ destination/
building file list ... done
created directory destination
./
file2.txt
dir1/
dir1/dir2/
dir1/dir2/file3.txt
dir3/

Verify the desitination directory to make sure the files and directories listed in the exclude-list.txt file is not backed-up.

$ find destination
destination
destination/file2.txt
destination/dir1
destination/dir1/dir2
destination/dir1/dir2/file3.txt
destination/dir3

Setting up PEAP wireless network from command line in Ubuntu 14.04

The wireless in my University uses PEAP.

I can access the internet from graphic interface, but need to input password all the time. I know it’s a fixed bug on Ubuntu 14.04 (https://bugs.launchpad.net/ubuntu/+source/network-manager-applet/+bug/1104476), and Ubuntu16.04 can login the network automatically.

I like to work in command line mode. Is it cooler? LOL.

I first linked to the wireless network in graphic interface, and  installed CUDA then I found I got a black screen. It was really a waste of time to solve NVIDIA driver problem. So I simply used command line. I know this is not a spirit of a geek : )

Anyhow, I found CUDA worked well in command line, that was enough.

The thing I need later was network. I got the solution form the following link. Just modify your file as the answer from keith and Aditya. Your network will automatically work when you enter Ubuntu.

http://askubuntu.com/questions/279762/cant-connect-to-wpa2-enterprise-peap