Sometimes, it is very helpful to have a timestamp on bash history, that way it’s easier to know the exact time a command was executed.
To put a timestamp on history, run the following command;
HISTTIMEFORMAT="%d/%m/%y %T "
That’s all. Next time you run the history command, the history will display with timestamp.
Hope someone finds it useful.
Trying to shutdown an old web server from the late 1990’s that had it’s guts transplanted onto a newer system around 2003 and again around 2009. As you can imagine there are accounts and files that are like those items in your junk drawer, they beg the question…why is this here?!
In an attempt to determine last use of accounts we combined some log analysis with some unix timestamp forensics to prove that no one really needs this anymore!
The log analysis was pretty easy, track non-robot traffic to determine which accounts were being accessed and at what frequency and volume. The timestamp wasn’t difficult just had to isolate which files we wanted to analyze. Using the `stat`, `find` and/or the `ls` commands make this easy. In case you are not aware of this Linux/Unix stores a number of timestamps for each file. These timestamps store when any file or directory was last accessed (read from or written to), changed (file access permissions were changed) or modified (written to).
Three times tracked for each file in Linux/Unix are:
Aside from using atime, ctime or mtime, the easiest way to get the information we are looking for is using the `stat` command:
# stat /home/myhome/file1 File: `/home/myhome/file1' Size: 1498906 Blocks: 2928 IO Block: 4096 regular file Device: fd01h/64769d Inode: 3414009 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 500/ myhome) Gid: ( 500/ users) Access: 2016-01-26 12:53:01.309089993 -0500 Modify: 2013-07-15 10:28:05.241847000 -0400 Change: 2013-07-15 10:28:05.315848001 -0400
If you are looking for a large set of files that have been accessed/modified/changed before or after a specific date then using the `find` command is your best bet.
For single files or a small set of files the `ls` command is probably easier.
For information on how to use atime, ctime and mtime with `find` and `ls` refer to the man page for the specific command.
Disk block size on Linux for the ext2/3/4 file system does not affect the amount of data written to disk. Apparently, all of the allowed values for blocksize (1024, 2048 or the default of 4096 bytes) result in data of 4096 bytes being written to disk.
Cacti is a network monitoring program which gathers statistics every minute or so, and writes them the disk. When monitoring large systems, you may find that volume disk IO can be a problem.
Cacti writes the its data to RRD files (Round-Robin Databases). Each RRD holds related values from a piece of data equipment. These values may be stored for every minute, and then aggregated every 30 minutes and 24 hours, for example. As a result, in an RRD of one megabyte, only a handful bytes are updated every minute (or so) interval.
Unfortunately, although only a few bytes are updated, disk IO happens in much larger blocks. Thus diskIO can be 100 times or more of the theoretical minimum.
To help with this, I attempted to verify that a reducing the blocksize on a filesystem will reduce the IO. This failed. Modifying the blocksize had no effect on IO.
A file system was created to test each blocksize, with a command similar to the following
mke2fs -T ext4 -b 4096 /dev/sdb1
where /dev/sdb1 is the file partition and 4096 is a blocksize (the others were 2048 and 1024).
The ‘dd’ command was used to create an initial 1MB file filled with null chars. This file was used to write to in the test.
dd if=/dev/zero of=test.fil bs=1024 count=1024
The actual test was done with this short Python program:
import sys
import mmap
import random
NSIZE = 0x100000 # File size
fname = "test.fil" # File name
# Open mmap file
f = open(fname,"r+b")
mm = mmap.mmap(f.fileno(), 0)
# Initialize random number generator object
r = random.Random()
# Get number of bytes to write into memory
n = int(sys.argv[1]) # Read command line
for i in range(n):
pos = r.randint(0,NSIZE-1) # Choose memory location at random
mm[pos] = 'X' # Place X character onto disk
mm.close()
f.close()
The Python program was run with values for n of 0, 1, 2, 3, while watching the output from iostat to determine the number of KBytes written to disk. Here’s an example of using iostat:
> iostat -p sdb 5 Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn sdb 1.00 0.00 1.60 0 8 sdb1 1.00 0.00 1.60 0 8 avg-cpu: %user %nice %system %iowait %steal %idle 3.40 0.00 1.30 1.70 0.00 93.60 Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn sdb 1.20 0.00 2.40 0 12 sdb1 1.20 0.00 2.40 0 12 avg-cpu: %user %nice %system %iowait %steal %idle 4.41 0.00 1.30 0.50 0.00 93.79 Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn sdb 1.40 0.00 3.20 0 16 sdb1 1.40 0.00 3.20 0 16
Here “-p sdb” tells iostat to only report on the device /dev/sdb, and the “5″ tells iostat to display every 5 seconds.
While “iostat” is running, the test Python program is run. Iostat will show how many Kbytes are written to disk.
testwrite.py 1; sync # Wait five seconds testwrite.py 2; sync # Wait five seconds testwrite.py 3; sync
The lines above write 1, 2 and 3 bytes to disk. By waiting 5 seconds between each command, iostat will show the amount of Kbytes written for each one — the iostat output shown above is from a test. It shows that 8, 12 and 16 kilobytes were written to disk when 1, 2 and 3 bytes are written from the program. There is a constant amount of overhead of 6 KB for each write, which means that each bytes written to by the test program resulted in 4KB written to disk. This value persisted even when the filesystem blockwas was 1024, 2048 and 4096 (in each case, only the overhead grew, but not the amout of data written disk disk per byte written from the program).
Everyone knows that gzip’ing a file will usually make it smaller, but gzip’ing it again will not. In other words, once you’ve compressed a file, it won’t compress further. But here’s an exception.
If you make a file of identical bytes, it can be gzip’ed several times and still become smaller. In the example below, we made a 2GB file which consisted of identical bytes, all equal 0. The first compression reduced the size by a factor of 1024. The second by a factor of 602. Next, by a factor of 14. The next compression, not shown, grew the file by about 20 bytes. The total compression, over the three generations, was a factor of 9,023,041 !
> dd if=/dev/zero of=2gb.zero bs=1024576 count=2048 > gzip -c 2gb.zero > 2gb.zero.gz > gzip -c 2gb.zero.gz > 2gb.zero.gz.gz > gzip -c 2gb.zero.gz.gz > 2gb.zero.gz.gz.gz > ls -lh 2gb.zero 2.0G 2gb.zero 2.0M 2gb.zero.gz 3.4K 2gb.zero.gz.gz 328B 2gb.zero.gz.gz.gz
mailservertest can be used to make testing mail servers a bit easier. It is a short script written in Python. Here a few use cases.
You’ve just restarted a number of mail servers, and you want to verify that they are delivering mail
mailservertest my.email@example.com mail1.example.com mail2 mail3
This will send a test email to my.email@example.com through the four mail servers mail1, mail2, mail3 and mail4. Note that the full domain name needs to be specified for the first mail server mail1.example.com; afterwards the ‘example.com’ will be appended to the remaining mail servers. This will save you some typing.
When you run the command, you will see output like this:
mail1.example.com ... Connecting to server ... OK ... Sending the "helo" command ... OK ... Sending the email ... OK mail2.example.com ... Connecting to server ... OK ... Sending the "helo" command ... OK ... Sending the email ... OK mail3.example.com ... Connecting to server ... OK ... Sending the "helo" command ... OK ... Sending the email ... OK
The script verifies each step of the mail process for each server. Thus, you can see where in the process a problem is occurring. The -q option will turn off these messages, but error messages (if any) will still be shown.
The emails you receive will look like this
From: my.email@example.com Subject: MAILTEST: mail1 16:28:29 2013-12-08 To: my.email@example.com This is a test email Tag : MAILTEST: Date : 16:28:29 2013-12-08 All Recipients : my.email@example.com All Servers : mail1.example.com mail2.example.com mail3.example.com This Server : mail3.example.com ---------------------------------------------------- Sent from the script 'mailservertest' on the host 'Your-Host'
The Subject line was chosen to make it as useful as possible when viewed in an email client; it shows the server that sent the message and the time that the script was run. The “tag” MAILTEST can be customized using the -t option.
You need to verify that your mail server rejects invalid addresses during the initial connection.
If your mail server should reject email for invalid addresses, you can verify that with this command (shown with it’s output).
mailservertest -q noone@example.com mail1.example.com
ERROR: {'noone@example.com': (550, '5.1.1 <noone@example.com>: Recipient address rejected: User unknown in virtual alias table')}
First note that the -q option has removed the status output shown in the previous example. Only the error message is shown; and it verifies that the address is unknown.
You want to generate and inspect a bounce message
You may be able to generate an email bounce message by sending an email through one mail server functioning as an MTA (mail transfer agent) to a second mail server functioning as a MDA (mail delivery agent). If the MTA passes your message to the MDA, and the recipient address is invalid, the MDA will return a bounce message to the return address. Since in this case the recipient address and the return address must be different, we will need to use the -f option to mailservertest. The -f option sets the return address. By default, the recipient and return address are the same; this worked fine for the previous example, but not now.
mailservertest -f my-email@example.com broken-email@other.com mail.example.com
Here, broken-email@other.com is an invalid email address. We first send the email to our MTA mail.example.com. It will send the email to whatever mail server handles other.com. The email bounce will be sent to our email address my-email@example.com. Below is an example of what is returned.
Delivery has failed to these recipients or groups: broken-email@other.com The e-mail address you entered couldn't be found. Please check the recipient's e-mail address and try to resend the message. If the problem continues, please contact your helpdesk. Diagnostic information for administrators: Generating server: CASHUBE.grove.ad.other.com broken-email@other.com #550 5.1.1 RESOLVER.ADR.RecipNotFound; not found ##rfc822;broken-email@other.com Original message headers: Received: from mail.example.com (192.168.25.234) by CashubE.grove.ad.other.com (192.168.30.167) with Microsoft SMTP Server id 14.2.347.0; Sun, 8 Dec 2013 17:25:49 -0500 Received: from localhost (host.example.com [192.168.80.33]) by mail.example.com (Postfix) with SMTP id 2F1007F44 for <broken-email@other.com>; Sun, 8 Dec 2013 17:25:49 -0500 (EST) From: "my-email@example.com" <my-email@example.com> Subject: MAILTEST: mail 17:25:49 2013-12-08 To: <broken-email@other.com> Date: Sun, 8 Dec 2013 17:25:49 -0500 MIME-Version: 1.0 Content-Type: text/plain Message-ID: <e91f8c64-57e5-47db-a2bc-9857ce0c04f3@CASHUBE.grove.ad.other.com> Return-Path: my-email@example.com Reporting-MTA: dns;CASHUBE.grove.ad.other.com Received-From-MTA: dns;mail.example.com Arrival-Date: Sun, 8 Dec 2013 22:25:49 +0000 Original-Recipient: rfc822;broken-email@other.com Final-Recipient: rfc822;broken-email@other.com Action: failed Status: 5.1.1 Diagnostic-Code: smtp;550 5.1.1 RESOLVER.ADR.RecipNotFound; not found ForwardedMessage.eml Subject: MAILTEST: mail 17:25:49 2013-12-08 From: "my-email@example.com" <my-email@example.com> Date: 12/08/2013 05:25 PM To: <broken-email@other.com> This is a test email Tag : MAILTEST: Date : 17:25:49 2013-12-08 All Recipients : broken-email@other.com All Servers : mail.example.com This Server : mail.example.com ---------------------------------------------------- Sent from the script 'mailservertest' on the host 'host'
–
mailservertest can be used to make testing mail servers a bit easier. It is a short script written in Python. Here a few use cases.
You’ve just restarted a number of mail servers, and you want to verify that they are delivering mail
mailservertest my.email@example.com mail1.example.com mail2 mail3
This will send a test email to my.email@example.com through the four mail servers mail1, mail2, mail3 and mail4. Note that the full domain name needs to be specified for the first mail server mail1.example.com; afterwards the ‘example.com’ will be appended to the remaining mail servers. This will save you some typing.
When you run the command, you will see output like this:
mail1.example.com ... Connecting to server ... OK ... Sending the "helo" command ... OK ... Sending the email ... OK mail2.example.com ... Connecting to server ... OK ... Sending the "helo" command ... OK ... Sending the email ... OK mail3.example.com ... Connecting to server ... OK ... Sending the "helo" command ... OK ... Sending the email ... OK
The script verifies each step of the mail process for each server. Thus, you can see where in the process a problem is occurring. The -q option will turn off these messages, but error messages (if any) will still be shown.
The emails you receive will look like this
From: my.email@example.com Subject: MAILTEST: mail1 16:28:29 2013-12-08 To: my.email@example.com This is a test email Tag : MAILTEST: Date : 16:28:29 2013-12-08 All Recipients : my.email@example.com All Servers : mail1.example.com mail2.example.com mail3.example.com This Server : mail3.example.com ---------------------------------------------------- Sent from the script 'mailservertest' on the host 'Your-Host'
The Subject line was chosen to make it as useful as possible when viewed in an email client; it shows the server that sent the message and the time that the script was run. The “tag” MAILTEST can be customized using the -t option.
You need to verify that your mail server rejects invalid addresses during the initial connection.
If your mail server should reject email for invalid addresses, you can verify that with this command (shown with it’s output).
mailservertest -q noone@example.com mail1.example.com
ERROR: {'noone@example.com': (550, '5.1.1 <noone@example.com>: Recipient address rejected: User unknown in virtual alias table')}
First note that the -q option has removed the status output shown in the previous example. Only the error message is shown; and it verifies that the address is unknown.
You want to generate and inspect a bounce message
You may be able to generate an email bounce message by sending an email through one mail server functioning as an MTA (mail transfer agent) to a second mail server functioning as a MDA (mail delivery agent). If the MTA passes your message to the MDA, and the recipient address is invalid, the MDA will return a bounce message to the return address. Since in this case the recipient address and the return address must be different, we will need to use the -f option to mailservertest. The -f option sets the return address. By default, the recipient and return address are the same; this worked fine for the previous example, but not now.
mailservertest -f my-email@example.com broken-email@other.com mail.example.com
Here, broken-email@other.com is an invalid email address. We first send the email to our MTA mail.example.com. It will send the email to whatever mail server handles other.com. The email bounce will be sent to our email address my-email@example.com. Below is an example of what is returned.
Delivery has failed to these recipients or groups: broken-email@other.com The e-mail address you entered couldn't be found. Please check the recipient's e-mail address and try to resend the message. If the problem continues, please contact your helpdesk. Diagnostic information for administrators: Generating server: CASHUBE.grove.ad.other.com broken-email@other.com #550 5.1.1 RESOLVER.ADR.RecipNotFound; not found ##rfc822;broken-email@other.com Original message headers: Received: from mail.example.com (192.168.25.234) by CashubE.grove.ad.other.com (192.168.30.167) with Microsoft SMTP Server id 14.2.347.0; Sun, 8 Dec 2013 17:25:49 -0500 Received: from localhost (host.example.com [192.168.80.33]) by mail.example.com (Postfix) with SMTP id 2F1007F44 for <broken-email@other.com>; Sun, 8 Dec 2013 17:25:49 -0500 (EST) From: "my-email@example.com" <my-email@example.com> Subject: MAILTEST: mail 17:25:49 2013-12-08 To: <broken-email@other.com> Date: Sun, 8 Dec 2013 17:25:49 -0500 MIME-Version: 1.0 Content-Type: text/plain Message-ID: <e91f8c64-57e5-47db-a2bc-9857ce0c04f3@CASHUBE.grove.ad.other.com> Return-Path: my-email@example.com Reporting-MTA: dns;CASHUBE.grove.ad.other.com Received-From-MTA: dns;mail.example.com Arrival-Date: Sun, 8 Dec 2013 22:25:49 +0000 Original-Recipient: rfc822;broken-email@other.com Final-Recipient: rfc822;broken-email@other.com Action: failed Status: 5.1.1 Diagnostic-Code: smtp;550 5.1.1 RESOLVER.ADR.RecipNotFound; not found ForwardedMessage.eml Subject: MAILTEST: mail 17:25:49 2013-12-08 From: "my-email@example.com" <my-email@example.com> Date: 12/08/2013 05:25 PM To: <broken-email@other.com> This is a test email Tag : MAILTEST: Date : 17:25:49 2013-12-08 All Recipients : broken-email@other.com All Servers : mail.example.com This Server : mail.example.com ---------------------------------------------------- Sent from the script 'mailservertest' on the host 'host'
—
Setting up a new linux workstation and after installing the Citrix Receiver and attempting to start a module (Outlook) I got the dread SSL error:
Having run into this every 6 months or so I decided it was time to jot down the fix. The problem is the install does not install the Citrix SSL certificate into the browsers trusted certificate cache. So here is the solution for Firefox…
To prevent the SSL error 61 when accessing remote sessions:
Make Firefox’s certificates accessible to Citrix,
sudo ln -s /usr/share/ca-certificates/mozilla/* /opt/Citrix/ICAClient/keystore/cacerts
That’s it, you should be good to go!
You want to redirect all requests to your web server to a single page on the server.
Note that redirection to an entirely different server is trivial with RedirectMatch. You can redirect all traffic to http://my-server.org to http://other-server.org by adding this rule to the Apache config files for http://my-server.org
# Redirecting to another server - WRONG! Not our goal. RedirectMatch ^.*$ http://other-server.org/
However, we want to redirect all traffic to one local URL, you might try this
# Redirecting to this server - WRONG! Creates infinite redirection. RedirectMatch ^.*$ http://my-server.org/new.html
This will causes an infinite redirection loop, because every redirection to http://my-server.org/new.html will trigger the RedirectMatch rule.
Use a negative lookahead in the regular expression. The following configuration will work
# Redirecting to this server - CORRECT! RedirectMatch ^(?!/new.html)$ http://my-server.org/new.html
The negative lookahead requires that the requested URL not match /new.html. This prevents the infinite redirection.
MySQL supports session encryption using TLS. Here’s how to configure your server and client to use it.
To start, you will need a server SSL certificate file and a key file, and a file containing the certificate that signed your cert. In the MySQL configuration file /etc/my.cnf or /etc/mysql/my.cnf, add these three lines to both the [mysqld] and [mysqld_safe] sections,
ssl-ca=SIGNING-CERT-FILE ssl-cert=CERT-FILE ssl-key=KEY-FILE
Restart your server so this new configuration will take effect.
You have two choices here.
Add the following line under the [client] section
ssl-ca=SIGNING-CERT-FILE
and all subsequent network traffic when using the mysql command-line client will encrypted. The SIGNING-CERT-FILEis the same as above.
my.cnf file if you run the client like this
mysql --ssl-ca=SIGNING-CERT-FILE ...
First, you will need to configure my.cnf as above in the section “Using MySQL Command-Line Client”. Below is an example of how to call Perl’s DBI packag using DBI->connect with the mysql_ssl option
$handle = DBI->connect( "dbi:mysql:DB_NAME:DB_HOST:mysql_ssl=1", DB_USER, DB_PASSWORD );
Replace DB_NAME, DB_HOST, DB_USER, DB_PASSWORD with the database name, host, user and user’s password. Warning: If the database does not support SSL, the connection will still succeed, but it will be plain text.
You will need a copy of cert for the signing authority of the MySQL server’s cert, as in previous examples. The difference here is that Python will read the signing authority cert directly, and not via the MySQL my.cnf file. We use the Python’s MySQLdb module to connect to MySQL. Here’s is an example
import MySQLdb
dbh = MySQLdb.connect(
host=DB_HOST,
user=DB_USER,
passwd=DB_PASS,
db=DB_NAME,
ssl={"ca":"SIGNING-CERT-FILE"}
)
Warning: If the database does not support SSL, the connection will still succeed, but it will be plain text.
The only way to verify that your connection is using SSL is to sniff the traffic on the server or client using tcpdump, like this
tcpdump -nn -s2048 -X host CLIENT-OR-SERVER
where CLIENT-OR-SERVER is the IP address of the MySQL client if you are listening on the server, and vice versa.
For anyone that would like to stay in touch with me beyond my last day at UConn: