Monday, June 28, 2010

debugging and monitoring database usage on postgresql

this is a query that tells you the running queries on a database, the time they have been running, the process id (pid; in case you want to kill it).... it uses the table pg_stat_activity

UPDATE FOR POSTGRES 9.4: some fields have changed name : / procpid is now pid, and current_query is now query... if it still throughs error check the fields with \d
pg_stat_activity

here it is:

select pid, now() - query_start, trim(current_query), datname, usename, waiting from pg_stat_activity where query not ilike '%IDLE%' order by query_start;

this was told to me by a friend (thanks José Zap), i don't know where did he get it from...


if you see a query having trouble, use EXPLAIN <query> to check and see what is postgresql actually doing, try to avoid SEQuential SCANS as these are much slower... create indexes as necessary... watch out don't overcreate them ; )

Saturday, June 19, 2010

How to Find All Unread Messages in Gmail

taken from: http://email.about.com/od/gmailtips/qt/et_find_unread.htm 

Type "label:unread"


 

How to Find All Unread Messages in Gmail

 An Inbox, a huge Archive, starred mail, chats, Spam and all kinds of labels: ways to view mail abound in Gmail.

Where, though, is the unread mail — all the unread mail, be it in the archive or the Inbox or anywhere else, and only the unread mail? Fortunately, "unread" is an attribute that messages in Gmail carry like a label. This means you can search — and find — these unread emails easily.

Find All Unread Messages in Gmail

To view all (and only) unread messages in your Gmail account:
  • Type "label:unread" (not including the quotation marks) in the Gmail search field.
  • Click Search Mail.
If you do this search repeatedly or generally appreciate brevity, you can use "l:unread" or even "l:^u" instead of "label:unread".
Of course, you can combine the quest for unread messages with others:
  • "l:^u from:tim" finds all unread messages from "tim",
  • "l:^u l:^t" finds all unread starred mail, and
  • "l:^u l:^k subject:hi" finds all unread messages with "hi" in the subject that are in the Trash.

 

Wednesday, June 16, 2010

converting K, M and G to numbers in excel

i'm not sure if the command 'du' in linux to see Disk Usage has an option to show it in Bytes.

I normally use 'du -h' which shows it in "human readable format", but that show it with K for Kilobytes, "M"egabytes and "G"igabytes...

I needed to compare sizes of directories in Excel... so I did this formula for converting the letter to their equivalente numeric. it's in spanish, because my excel is in spanish, and excel has that annoying feature which translates function names !

=SI(ESNUMERO(ENCONTRAR("K";A4692));SUSTITUIR(A4692;"K";"")*1000;SI(ESNUMERO(ENCONTRAR("M";A4692));SUSTITUIR(A4692;"M";"")*1000000;SI(ESNUMERO(ENCONTRAR("G";A4692));SUSTITUIR(A4692;"G";"")*1000000000;"")))

Monday, June 14, 2010

exim exim4 configuration smtp_accept_queue_per_connection

I was getting the warning:  

no immediate delivery: more than 10 messages received in one connection

It was driving me crazy because I needed to send more mails, faster... ; )

I suffered to find this one out, and it was a stupid stupid detail....

I did:

vim /etc/exim4/update-exim4.conf.conf

and added the line at the end of the file:

smtp_accept_queue_per_connection=100

then reloaded exim's configuration

 $ sudo /etc/init.d/exim4 reload
 * Reloading exim4 configuration files   [ OK ]

the tricky part: WATCH OUT !! NO SPACES BETWEEN after or before the "=", they will cause an error

Tuesday, June 8, 2010

spf ptr reverse dns lookup

we have an application that sends mails to users.

emails sent to hotmail were being automatically redirected to the SPAM folder.

after some little research, we found out that we had to configure an SPF records in out DNS manager...

we did this, and apparently didn't work.

we noticed then, that making a reverse dns lookup to out domain returned a somewhat strange name. we included this name on the PTR part of the record, und alles ok.

we then noticed that /etc/hosts file had this strange name written into it. so we changed it to what it must say: our domain name.


there are many tools to help you write an SPF record.... just google it

Friday, June 4, 2010

enforce secure password to linux users

i'm trying libpam-cracklib....

it's kind of straight forward... takes just a few seconds

kind of good... taken from: http://www.deer-run.com/~hal/sysadmin/pam_cracklib.html


Linux Password Security with pam_cracklib

Hal Pomeranz, Deer Run Associates

Standard Unix reusable passwords are not really a good authentication system. However, the costs associated with migrating to an alternate authentication system such as two-factor token authentication or smartcard-based systems are too high for most enterprises. So sites are generally left with the "lowest common denominator" option provided by their vendors.

In order to improve the security of standard reusable passwords, "best practices" tell us to require users to change their passwords on a regular basis, enforce minimum lengths and good "rules" for new passwords (such as requiring mixed case and non-alphanumeric characters), and even keep a "history" of previous user passwords so that users don't "repeat". Interestingly, Unix systems have typically lagged behind other operating systems in providing this functionality--particularly when it comes to rule-based systems for requiring strong passwords, as well as functionality for keeping password history. In an effort to address this shortcoming, the PAM module pam_cracklib was developed for Linux systems.

I admit that for the longest time I thought pam_cracklib was just about useless. But it turns out that it's not useless, it's just really poorly documented. In an effort to correct this problem, I present the following article based on my research with the existing documentation, the pam_cracklib source code (when in doubt, read the source), and my trusty Knoppix (Debian) GNU/Linux system.

Enabling pam_cracklib

The pam_cracklib module is enabled via the system's standard PAM configuration interface. On Debian systems, this is the /etc/pam.d/common-password file (but it's /etc/pam.d/system-auth on RedHat-derived systems--can't we all just get along?). The typical configuration looks something like this:

password required pam_cracklib.so retry=3 minlen=6 difok=3
password required pam_unix.so md5 use_authtok

The first line enables the pam_cracklib module and sets several module parameters. "retry=3" means that users get three chances to pick a good password before the passwd program aborts. Users can always re-run the passwd program and start over again, however. "minlen=6" sets the minimum number of characters in the password. Actually, since Linux systems generally use MD5 password hashes, which are not limited to 8 character passwords like the old DES56 hashes, you probably want to think about increasing the "minlen" parameter to something a bit longer. We'll come back to this notion a bit later in the article. "difok=3" sets the minimum number of characters that must be different from the previous password. If you increase "minlen", you may also want to increase this value as well.

The second line invokes the standard pam_unix module. The "md5" argument here is what enables standard Linux MD5 password hashes, though you have the option of using old-style DES56 hashes for backwards compatibility with legacy Unix systems. "use_authtok" tells pam_unix to not bother doing any of its own internal password checks, which duplicate many of the checks in pam_cracklib, but instead accept the password that the user inputs after it's been thoroughly checked by pam_cracklib.

Simple Checks

By default, pam_cracklib performs a number of basic checks on the new password:

  • Is the new password just the old password with the letters reversed ("password" vs. "drowssap") or rotated ("password" vs. "asswordp")?
  • Does the new password only differ from the old one due to change of case ("password" vs. "Password")?
  • Are at least some minimum number of characters in the new password not present in the old password? This is where the "difok" parameter comes into play.

These are the same checks you get in the pam_unix module if you turn on the "obscure" flag, but since we're already using pam_cracklib we don't need to do this.

Length and Strength

While the "minlen" parameter controls the minimum password length, things are not as simple as they might appear. This is because pam_cracklib combines the notion of password length with password "strength" (the use of mixed-case and non-letter characters).

"minlen" is actually the minimum required length for a password consisting of all lower-case letters. But users get "length credits" for using upper- and lower-case letters, numbers, and non-alphanumeric characters. The default is normally that you can only get a maximum of "1 credit" for each type of character. So if the administrator sets "minlen=12", a user could still have an 8 character password if they used all four types of characters. Actually, since using a lower-case letter gets you a credit, the real minimum length for an all lower-case password is minlen-1.

The maximum credit for any particular class of characters is actually customizable. The four parameters "lcredit", "ucredit", "dcredit", and "ocredit" are used to set the maximum credit for lower-case, upper-case, numeric (digit), and non-alphanumeric (other) characters, respectively. For example, you could add the following parameters on the pam_cracklib line in the /etc/pam.d/common-password file:

lcredit=0 ucredit=1 dcredit=1 ocredit=2

In other words, lower-case characters aren't special at all, so you get no credit there. On the other hand we give extra credit if the user puts two or more non-alphanumeric characters in their password. One point is still the max credit for upper-case characters and numbers. Note that no matter what you set "minlen" to and no matter how many "credits" you give to your users, pam_cracklib will never let users pick passwords with less than six characters--this is a hard-coded internal minimum.

Play around with these values and find something that makes sense for your site, but as a starting point I might recommend "minlen=12 difok=4" for machines using MD5 password hashes. This means that the smallest password a user could have is 8 characters, and that's only if they use all four character sets.

Dictionary Checks

pam_cracklib also checks the user's password against it's own internal dictionaries of easily guessed passwords. On Debian systems, pam_cracklib's dictionaries live in /var/cache/cracklib and are rebuilt nightly by the
/usr/sbin/update-cracklib script. Other Linux distros may have other mechanisms for updating the dictionaries (as far as I can tell, RedHat doesn't provide any tools for doing this).

The update-cracklib script searches a number of directories for input files, including /usr/local/dict and /usr/local/share/dict. So adding your own words is as easy as putting them in a file in one of these directories and running update-cracklib or waiting for cron to do it for you. Note that if you want to add other directories to update-cracklib's search path, you can do this by modifying the /etc/cracklib/cracklib.conf file (at least on Debian systems).

Password "History"

pam_cracklib is capable of consulting a user's password "history" and not allowing them to re-use old passwords. However, the functionality for actually storing the user's old passwords is enabled via the pam_unix module.

The first step is to make sure to create an empty /etc/security/opasswd file for storing old user passwords. If you forget to do this before enabling the history feature in the PAM configuration file, then all user password updates will fail because the pam_unix module will constantly be returning errors from the password history code due to the file being missing.

Treat your opasswd file like your /etc/shadow file because it will end up containing user password hashes (albeit for old user passwords that are no longer in use):

touch /etc/security/opasswd
chown root:root /etc/security/opasswd
chmod 600 /etc/security/opasswd

Once you've got the opasswd file set up, enable password history checking by adding the option "remember=<x>" to the pam_unix configuration line in the /etc/pam.d/common-password file. Here's how I have things set up on my Knoppix machine:

password required pam_cracklib.so retry=3 minlen=12 difok=4
password required pam_unix.so md5 remember=12 use_authtok

The value of the "remember" parameter is the number of old passwords you want to store for a user. It turns out that there's an internal maximum of 400 previous passwords, so values higher than 400 are all equivalent to 400. Before you complain about this limit, consider that even if your site forces users to change passwords every 30 days, 400 previous passwords represents over 30 years of password history. This is probably sufficient for even the oldest of legacy systems.

Once you've enabled password history, the opasswd file starts filling up with user entries that look like this:

hal:1000:<n>:<hash1>,<hash2>,...,<hashn>

The first two fields are the username and user ID. The <n> in the third field represents the number of old passwords currently being stored for the user--this value is incremented by one every time a new hash is added to the user's password history until <n> ultimately equals the value of the "remember" parameter set on the pam_unix configuration line. <hash1>,<hash2>,...,<hashn> are actually the MD5 password hashes for the user's old passwords.

Password Expiration

At this point you may be wondering how to get the system to automatically force users to change their password after some period of time. This is not actually the job of pam_cracklib. Instead, these parameters are set in the /etc/login.defs file on most Linux systems. PASS_MAX_DAYS is how often users have to change their passwords. PASS_MIN_DAYS is how long a user is forced to live with their new password before their allowed to change it again. PASS_WARN_AGE is the number of days before the password expiration date that the user is warned that their password is about to expire. The choice of values for these parameters is entirely dependent on site policy.

Note that these parameters are only applied to new accounts created with the default system useradd program. If you use some other mechanism for creating accounts on the system, then you'll have to use the chage command (this is not a typo) to manually set these parameters on your user accounts. And if you use a naming service such as LDAP or NIS for account management, then you're completely on your own.

By the way, if you've ever wondered what all those extra fields in the /etc/shadow file were for, the answer is that they store the password expiration/aging information for the user.

Conclusion

Hopefully this article sheds light on some of the more mysterious corners of pam_cracklib's functionality and the Linux password system in general. If pam_cracklib appears limited compared to the password enforcement routines on your particular operating system variant, consider that some Unix-like operating systems (such as Solaris--at least through Solaris 9) don't have any functionality of this type, other than the standard password expiration/aging routines.

About the Author

Hal Pomeranz (hal@deer-run.com) realizes that the wonderful thing about Linux is that you can always refer back to the source code, but would prefer to read a Unix manual page whenever possible.

Tuesday, June 1, 2010

turn on autoflush in perl

to automatically flush the results of the Print commands:


Turn on "autoflush":

Put this just after your #!/.../perl line.
$|++;





Turn on autoflush in a BEGIN block:
BEGIN { $| = 1 } taken from: http://www.perlmonks.org/?node_id=20590

cut command - listing linux users from /etc/passwd

I discovered an interesting command, 'cut'


$ cat /etc/passwd |grep "/home" |cut -d: -f1

It is here applied to list linux user with /bin/bash shell...

taken from: http://www.linuxquestions.org/linux/answers/Networking/How_to_list_all_your_USERs

As we all know the user list reside in /etc/passwd, so we can view the registered user by looking up at this file.
But now the fuss is that it also contains many other fields & machine trust accounts & inbuilt accounts.

So now we'll make a command of our own..

We'll start by
1.cat /etc/passwd

2.As we all know that by default all the users created will have their home directories in /home share
so we'll modify our command a bit by using grep.
Now it'll be
cat /etc/passwd | grep "/home"

3. Now we'll get all the user accounts which have their home share in /home.
But the only output we need is the list of users & nothing else.

4. So we'll modify our command again
cat /etc/passwd | grep "/home" |cut -d: -f1
Now what we have done is that we have piped the output of previous command to another variable "cut"
What we have done here is we have added
cut -d: -f1
-d: means delimiter :
-f1 means display first field of line i.e. username.

So final command is
Cat /etc/passwd |grep "/home" |cut -d: -f1

cancel script completely on ctrl-c

I found this question interesting: basically how to cancel completely a script and all child processes : You do this by creating a subro...