Wednesday, December 22, 2010

egrep to filter out lines in a text file NOT cointaining certain strings

this worked for me:

egrep '^str2|str1' file.txt

winscp delete after successful transfer (put/get)

I've been using WinScp and a script to compress files at maximum compressiĆ³n, name the file as name_YYMMDD-HHMM.rar, and send them to a remote server; then I wanted to delete local files after a successful transfer. Here's my batch file (it works on WinXP). :

;FOR /F "tokens=1-4 delims=/ " %%I IN ('DATE /t') DO SET mydate=%%K%%J%%I
;
;FOR /F "eol=; tokens=1-2 delims=: " %%a IN ('TIME /t') DO SET mytime=%%a%%b
;
;ECHO The value is "%mydate% %mytime%"
;
;set varname=myname-%mydate%-%mytime%.rar
;
;rar m -m5 %varname% "FolderWithFilesToCompress\*.jpg"
;echo %varname%

start /min winscp.exe /console /command "option batch on" "open lab@64.150.188.62" "option confirm off" "put -delete %varname%" "exit"


To call the batch file automatically in a minimized mode, I call from window's Task Scheduler this vbs script (the last 0 means minimized):

Set WinScriptHost = CreateObject("WScript.Shell")
WinScriptHost.Run Chr(34) & "C:\path\to\batch\mybatch.bat" & Chr(34), 0
Set WinScriptHost = Nothing

And here the documentarion on WinScp

Script Commands

In its scripting functionality, WinSCP supports set of commands described below.

You can see the very same help for the commands as shown here, if you type command help directly in console.

General Syntax

Command parameters that include space(s) have to be enclosed in double-quotes. To use double-quote literally, double it:

put "file with spaces and ""quotes"".html"

You can use environment variables in the commands, with syntax %NAME%1):

put "%FILE_TO_UPLOAD%"

You can reference script arguments (passed on command-line using parameter /parameter) using syntax %N%, where N is ordinal number of argument1):

put "%1%"

Note that WinSCP treats filenames in case sensitive manner. So even if your server treats filenames in case insensitive manner, make sure you specify case properly2).

Advertisements:

call

With SFTP and SCP protocols, executes arbitrary remote shell command. With FTP protocol, executes a protocol command.

call 

If current session does not allow execution of arbitrary remote command separate shell session will be automatically opened.

The command must not require user input.

Alias: !

XML Log Element: call

Examples:

call mysqldump --opt -u USERNAME --password=PASSWORD --all-databases > all_databases.sql
call gzip -c all_databases.sql > all_databases.gz

cd

Changes remote working directory for active session.

cd [  ]

If directory is not specified, changes to home directory.

Examples:

cd /home/martin
cd

chmod

Changes permissions of one or more remote files.

chmod   [  ... ]

mode can be specified as three or four-digit octal number.

Filename can be replaced with wildcard to select multiple files.

XML Log Element: chmod

Examples:

chmod 644 index.html about.html
chmod 1700 /home/martin/public_html
chmod 644 *.html

close

Closes session.

close [  ]

Closes session specified by its number. When session is not specified, closes currently selected session.

Examples:

close 1
close

exit

Closes all sessions and terminates the program.

Advertisements:
exit

Alias: bye

get

Downloads one or more files from remote directory to local directory.

get  [ [  ... ] \[  ] ]

Downloads one or more files from remote directory to local directory. If only one parameter is specified downloads the file to local working directory. If more parameters are specified, all except the last one specify set of files to download. The last parameter specifies target local directory and optionally operation mask to store file(s) under different name. Destination directory must end with backslash. Filename can be replaced with wildcard to select multiple files. To download more files to current working directory use .\ as the last parameter.

Use option command to set transfer options.

Alias: recv

Switches:

Switch Description
-delete Delete source remote file(s) after transfer.
-resume Automatically resume transfer if possible (SFTP and FTP protocols only). Cannot be combined with -append.
-append Append source file to the end of target file (SFTP protocol only). Cannot be combined with -resume.
-preservetime Preserve timestamp
-nopreservetime Do not preserve timestamp
-speed= Limit transfer speed

Effective options: transfer, confirm, exclude, include, reconnecttime

XML Log Elements: download, rm (with -delete)

Examples:

get index.html
get -delete index.html about.html .\
get index.html about.html d:\www\
get public_html/index.html d:\www\about.*
get *.html *.png d:\www\*.bak

See also synchronize if you need to transfer modified or non-existing files only.

help

Displays help for script commands.

help [  [  ... ] ]

Displays list of commands when no parameters are specified. Displays help for each command when some are specified.

Alias: man

Examples:

help ls
help

keepuptodate

Watches for changes in local directory and reflects them on remote one.

keepuptodate [  [  ] ]

When directories are not specified, current working directories are synchronized. To stop watching for changes press Ctrl-C.

Note: Overwrite confirmations are always off for the command.

Switches:

Switch Description
-delete Delete obsolete files
-permissions= Set permissions (SFTP and SCP protocols only)
-nopermissions Keep default permissions
-speed= Limit transfer speed

Effective options: transfer, exclude, include, reconnecttime

XML Log Elements: upload, touch, chmod (with -permissions), rm (with -delete)

Examples:

keepuptodate -delete
keepuptodate d:\www /home/martin/public_html

lcd

Changes local working directory for all sessions.

lcd 

Example:

lcd d:\

lls

Lists the contents of local directory.

Advertisements:
lls [  ]\[  ]

If directory is not specified, lists working directory. When wildcard is specified, it is treated as set of files to list. Otherwise, all files are listed.

Examples:

lls *.html
lls d:\
lls

ln

Creates remote symlink.

ln  

Alias: symlink

Example:

ln /home/martin/public_html www

lpwd

Prints current local working directory (valid for all sessions).

lpwd

ls

Lists the contents of specified directory.

ls [  ]/[  ]

Lists the contents of specified remote directory. If directory is not specified, lists working directory. When wildcard3) is specified, it is treated as set of files to list. Otherwise, all files are listed.

Alias: dir

XML Log Element: ls

Examples:

ls *.html
ls /home/martin
ls

mkdir

Creates remote directory.

mkdir 

XML Log Element: mkdir

Example:

mkdir public_html

mv

Moves or renames one or more remote files.

mv  [  ... ] [ / ][  ]

Destination directory or newname or both must be specified. Destination directory must end with slash. Operation mask can be used instead of new name. Filename can be replaced with wildcard to select multiple files.

Alias: rename

XML Log Element: mv

Examples:

mv index.html public_html/
mv index.html about.*
mv index.html public_html/about.*
mv public_html/index.html public_html/about.html /home/martin/*.bak
mv *.html /home/backup/*.bak

open

Establishes new connection.

open 
open [ sftp|ftp|scp:// ][ [ :password ] @ ] [ : ]

Establishes connection to given host. Use name of the stored session (to open session, stored in folder, use path syntax “folder/session”). You can also specify host, username, port and protocol directly.

Switches:

Switch Description
-privatekey= Private key path
-timeout= Server response timeout
-hostkey=“ Specifies fingerprint of expected SSH host key (or several alternative fingerprints separated by semicolon). It makes WinSCP automatically accept hostkey with the fingerprint. As the hostkey fingerprint contains spaces you need to enclose it in quotes. SFTP and SCP protocols only.
-certificate=“ Specifies fingerprint of expected SSL/TLS sertificate (or several fingerprints separated by semicolon). It makes WinSCP automatically accept certificate with the fingerprint. FTPS protocol only.
-passive Passive mode (FTP protocol only)
-implicit Implicit TLS/SSL (FTPS protocol only)
-explicitssl Explicit SSL (FTPS protocol only)
-explicittls Explicit TLS (FTPS protocol only)

XML Log Element: session

Examples:

open sftp://martin@example.com -hostkey="ssh-rsa 1024 xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"
open scp://test@example.com:2222 -privatekey=mykey.ppk
open ftps://martin@example.com -implicit -certificate="xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"
open martin@example.com
open example.com
open

option

Shows or sets value of script options.

option [ 

If no parameters are specified, lists all script options and their values. When one parameter is specified only, shows value of the option. When two parameters are specified sets value of the option. Initial values of some options are taken from application configuration, however modifing the options does not change the application configuration.

Options are:

Option Values and description
echo on|off
Toggles echoing of command being executed.
Commands affected: all
batch on|off|abort|continue
Toggles batch mode (all prompts are automatically replied negatively). When on, it is recommended to set confirm to off to allow overwrites. With abort script is aborted when any error occurs. With continue all errors are ignored.
Commands affected: nearly all
confirm on|off
Toggles confirmations (overwrite, etc.).
Commands affected: get, put
transfer binary|ascii|automatic
Transfer mode: binary, ascii (text), automatic (by extension).
Commands affected: get, put, synchronize, keepuptodate
exclude
include
clear | [;...]
Sets exclusion or inclusion masks (only one can be set at time).
Commands affected: get, put, synchronize, keepuptodate
reconnecttime off |
Sets time limit in seconds to try reconnecting broken sessions.
Commands affected: get, put, synchronize, keepuptodate

Aliases: ascii (for option transfer ascii), binary (for option transfer binary)

Examples:

option transfer
option confirm off
option include "*.html; */"
option exclude "*.tpl.php"
option exclude "*.mp3; *.mp4; *.lnk; *.exe; *.msi; My Pictures; My Music; My Videos;"
option

Note that resetting the same option overwrites previous values, it does not append.

put

Uploads one or more files from local directory to remote directory.

put  [ [  ... ] /[  ] ]

If only one parameter is specified uploads the file to remote working directory. If more parameters are specified, all except the last one specify set of files to upload. The last parameter specifies target remote directory and optionally operation mask to store file(s) under different name. Destination directory must end with slash. Filename can be replaced with Windows wildcard3) to select multiple files. To upload more files to current working directory use ./ as the last parameter.

Use option command to set transfer options.

Switches:

Switch Description
-delete Delete source local file(s) after transfer.
-resume Automatically resume transfer if possible (SFTP and FTP protocols only). Cannot be combined with -append.
-append Append source file to the end of target file (SFTP protocol only). Cannot be combined with -resume.
-preservetime Preserve timestamp
-nopreservetime Do not preserve timestamp
-permissions= Set permissions (SFTP and SCP protocols only)
-nopermissions Keep default permissions
-speed= Limit transfer speed

Alias: send

Effective options: transfer, confirm, exclude, include, reconnecttime

XML Log Elements: upload, chmod (with -permissions), touch (with -preservetime)

Examples:

put index.html
put -delete index.html about.html ./
put -permissions=644 index.html about.html /home/martin/public_html/
put d:\www\index.html about.*
put *.html *.png /home/martin/backup/*.bak

See also synchronize if you need to transfer modified files only.

pwd

Prints current remote working directory for active session.

pwd

rm

Removes one or more remote files.

rm  [  ... ]

If remote recycle bin is configured, moves file to the bin instead of deleting it. Filename can be replaced with wildcard to select multiple files.

XML Log Element: rm

Examples:

rm index.html
rm index.html about.html
rm *.html

rmdir

Removes one or more remote directories.

rmdir  [  ... ]

If remote recycle bin is configured, moves directory to the bin instead of deleting it.

XML Log Element: rm

Example:

rmdir public_html

session

Manages opened sessions.

session [  ]

Makes session specified by its number active. When session number is not specified, lists connected sessions.

Examples:

session 1
session

synchronize

Synchronizes directories.

synchronize local|remote|both [  [  ] ]

When the first parameter is local, changes from remote directory are applied to local directory. When the first parameter is remote, changes from the local directory are applied to the remote directory. When the first parameter is both, both local and remote directories can be modified.

When directories are not specified, current working directories are synchronized.

Note: Overwrite confirmations are always off for the command.

Switches:

Switch Description
-delete Delete obsolete files. Ignored for both.
-mirror Mirror mode (synchronize also older files). Ignored for both.
-criteria= Comparison criteria. Possible values are time, size, both and none. Ignored for both mode.
-permissions= Set permissions (SFTP and SCP protocols only)
-nopermissions Keep default permissions
-speed= Limit transfer speed

Effective options: transfer, exclude, include, reconnecttime

XML Log Elements: download (with local or both), upload (with remote or both), touch (with remote or both), chmod (with remote or both and -permissions), rm (with remote and -delete)

Examples:

synchronize remote -delete
synchronize both d:\www /home/martin/public_html
1) Generally do enclose reference to double-quotes to cope properly with spaces in its value.
2) This is important particularly for FTP sessions.
3) Windows wildcard supports * and ? only. It does not support all the features of file masks.
script_commands.txt · Last modified: 7 Nov 2010 by prikryl

Search

This page


Sunday, December 19, 2010

sed script to strip ^M characters

this may also be handy, to strip ^M characters:
(from http://www.computing.net/answers/unix/remove-m-and-line-feeds/6934.html)
anupam May 23, 2005 at 21:11:55 Pacific

Hi..to remove ^M, give the command :
sed -e 's/^M//g' filename
you will have to write the control-M as follows : first press control-V,then control-M,this will show the correct control-M on the command line.
For removing the linefeeds,try this :
sed -e 's/.$//g' filename
this will remove the last character from the line,which happens to be the line feed.

Saturday, December 18, 2010

installink XEphem on Ubuntu

had I found this link earlier, i wouldn't have wasted so much time and effort:

http://www.tc.umn.edu/~brams006/xephem_ubuntu.html

How to Compile XEphem on Ubuntu Linux

Following is a (hopefully) painless guide to compiling the excellent astronomy program named XEphem (http://www.clearskyinstitute.com/xephem/) on the Ubuntu linux distribution. Though we now have Stellarium and KStars, XEphem has some unique features and the source code is free for personal/educational use (check the URL for precise terms). In addition, it doesn't require a higher-end OpenGL graphics card. It'll run well on a fairly modest machine. I've tested these steps on Ubuntu 10.04 Lucid Lynx and they worked for me without any trouble.

(1) Preliminaries & Packages.

Use either "sudo apt-get install" or Synaptic to pull down (or verify that you already have) the following packages installed:

gcc
lesstif2-dev
libc6-dev
libxmu-dev
make

Since I embarked on this document (perhaps in 2005), I've noticed a couple changes with the Ubuntu packages. When you install the gcc compiler it sets up a symlink automatically from /usr/bin/gcc to the /usr/bin/gcc-{version number} executable. In the past, you may have needed to create this symlink yourself. For the newbies: 'cd /usr/bin', then 'ln -s gcc-4.4 gcc' or whatever was the gcc version.

If you'd rather compile with libmotif3 and libmotif3-dev (instead of the lesstif2 and lesstif2-dev respectively), you'll need to go into Synaptic (or edit /etc/apt/source.list manually), enable the Non-free Multiverse repository, and install those packages instead. There are some licensing politics surrounding motif.

In any case, I've successfully compiled XEphem 3.7.4 with gcc-4.x and the LGPL lesstif2/lesstif2-dev Ubuntu maintained packages, as well as the libmotif3/libmotif3-dev alternative.

(2) Get the XEphem source code.

Go here: http://www.clearskyinstitute.com/xephem/. Download it to a directory, for instance your user directory, and ungzip/untar it. It'll be in a directory named "xephem-3.7.4" (depending on version number).

(3) Make a change to the xephem-3.7.4/libz/Makefile

This file may initially be read-only, so be sure to chmod +w it if necessary. I noticed this issue with the source for XEphem 3.7.1 and 3.7.2 as well. There's a couple things missing from the makefile in the xephem-3.7.4/libz/Makefile file. Make sure the top of that file includes this:

CC = gcc
CLDFLAGS = -g
CFLAGS = $(CLDFLAGS) -Wall -O2
LDFLAGS =

And the bottom of the file needs to have "gcc" specified rather than cc.:

testzlib: testzlib.o libz.a
gcc $(LDFLAGS) -o testzlib testzlib.o libz.a

(4) Compile XEphem.

Whether you're using motif or lesstif, the XEphem makefiles expect some value for "MOTIF=" in order to compile. Go to the xephem-3.7.4/GUI/xephem directory. Issue this:

make MOTIF=/usr/lib/

XEphem should now successfully compile after a minute or so, depending on the speed of your hardware.

(5) Run and test.

Go to /xephem-3.7.4/GUI/xephem and type "./xephem" to execute. The first things you might want to do is to set the observer location to your own and confirm that your time/zone information is correctly indicated.

I like to create a KDE desktop icon for XEphem. From the KDE desktop, right-click and create a new link to an application. Go to the directory /xephem-3.7.4/GUI/xephem and select the xephem executable, change the icon to the XEphem.png file there. Then make sure the working directory is set to /xephem-3.7.4/GUI/xephem (or wherever you actually installed it).

As of Ubuntu 10.04 I decided to switch to Gnome. I see that the Gnome desktop application launcher doesn't include a "working directory" setting. Unless XEphem is executed from the path it resides in, some components may be improperly pathed. Being less experienced with Gnome, I'm not sure if this is the best approach, but it was awfully simple:

In the ~/xephem3.7.4/ directory I created a simple script called xephem.sh, and chmod +x to it. Here's the contents of the script:

cd /home/{my user directory}/xephem-3.7.4/GUI/xephem
./xephem

I then aimed the Gnome desktop launcher at this script, and used the icon here: /xephem-3.7.4/GUI/xephem/XEphem.png.

Enjoy!


The views and opinions expressed in this page are strictly those of the page author.
The contents of this page have not been reviewed or approved by the University of Minnesota.

missing files

when installing from source file, using make, and it says error "file not found" check apt-file utility for which package you need to install


FROM: http://superuser.com/questions/10997/find-what-package-a-file-belongs-to-in-ubuntu-debian

I frequently find myself missing a program, man page, or other file when working on my Ubuntu 8.04 system. Is there any simple way to look up what package contains a given file (whether it is installed already or not)? Maybe some obscure option for apt or dpkg?

link|flag




3 Answers

up vote 7 down vote accepted
apt-file search filename

or

apt-file search /path/to/file


if you need apt-file, use:

apt-get install apt-file

you may also want to update its database:

sudo apt-file update

running a windows scheduled task minimized

also you can do start /min \path\to\program.exe inside the batch, to run programs from the batch file in minimized mode


You could run it silently using a Windows Script file instead. The Run Method allows you running a script in invisible mode. Create a .vbs file like this one

Set WinScriptHost = CreateObject("WScript.Shell")
WinScriptHost.Run Chr(34) & "C:\Scheduled Jobs\mybat.bat" & Chr(34), 0
Set WinScriptHost = Nothing

and schedule it. The second argument in this example sets the window style. 0 means "hide the window."

Complete syntax of the Run method:

 object.Run(strCommand, [intWindowStyle], [bWaitOnReturn])

Arguments:

  • object: WshShell object.
  • strCommand: String value indicating the command line you want to run. You must include any parameters you want to pass to the executable file.
  • intWindowStyle: Optional. Integer value indicating the appearance of the program's window. Note that not all programs make use of this information.
  • bWaitOnReturn: Optional. Boolean value indicating whether the script should wait for the program to finish executing before continuing to the next statement in your script. If set to true, script execution halts until the program finishes, and Run returns any error code returned by the program. If set to false (the default), the Run method returns immediately after starting the program, automatically returning 0 (not to be interpreted as an error code).
link|flag


taken from: http://serverfault.com/questions/9038/run-a-bat-file-in-a-scheduled-task-without-a-window





cancel script completely on ctrl-c

I found this question interesting: basically how to cancel completely a script and all child processes : You do this by creating a subro...