Author: Saba, Mitch

Issues with Tomcat7 on RHEL7

On RHEL7, up until tomcat v7.0.59 Redhat maintained backwards compatibility with RHEL6 structures.  Once you upgrade to tomcat 7.0.70-x all that changes.  Two things specifically:

One big change is the use of $CATALINA_OPTS, if you have multiple lines defining various $CATALINA_OPT properties your installation will break.  Once you upgrade you must specify all of your $CATALINA_OPTS on a single line.  This has to do with the use of systemd, regardless all on one line or it won’t work.

Ensure tomcat7 is logging to /var/log/tomcat7/catalina.out

Create a new file /etc/rsyslog.d/tomcat.conf containing the next two lines:

programname,contains,"server" /var/log/tomcat/catalina.out
programname,contains,"server" ~

Afterwards, restart the rsyslog daemon:

service rsyslog restart

Those are the only issues I have found yet, I’ll update if more surface.



WordPress: Redirect HTTP to HTTPS in Apache

To redirect all of your HTTP traffic to HTTPS on WordPress installations using an Apache web server, add the following code to your .htaccess file. This is the recommended method for redirecting WordPress running on Apache.


RewriteCond %{SERVER_PORT} 80 
RewriteRule ^(.*)$$1 [R=301,L]

This should begin to work immediately.

Satellite Server Automated Reboot

Automating the patch cycle with Satellite v6 series is not all that difficult, with one gotcha, rebooting.  My initial setup had a test to see if there were any patches that would require a reboot and if so reboot.  However, the servers did not reboot as expected.  It seems the remote execution process was skipping the reboot command due to some delay.  The fix is to issue the following command which provides enough of a delay to have the reboot command picked up and executed.  The same holds true when using the shutdown command.

echo restart host && sleep 3


Repairing WordPress MySql Table corruption

We suffered a SAN outage which caused one of our wordpress servers to come back up with a corrupted table.


In the wordpress logs this presented as:

[error][client IP] Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer:

Not exactly clear error message.  Digging into the mysqld log was more illustrative of the problem:

[ERROR] /usr/libexec/mysqld: Table './db/wp_options' is marked as crashed and should be repaired

(The database name was changed to protect the innocent).

With the above error, I now understood the problem and how to approach fixing it.

# mysql -u root -p
Enter password:
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

mysql> connect db;

Connection id: 2244
Current database: db

mysql> select * from wp_options limit 1;
ERROR 145 (HY000): Table ‘./db/wp_options’ is marked as crashed and should be repaired
mysql> repair table wp_options;
| Table | Op | Msg_type | Msg_text |
| db.wp_options | repair | info | Wrong block with wrong total length starting at 52540 |
| db.wp_options | repair | warning | Number of rows changed from 314 to 313 |
| db.wp_options | repair | status | OK |
3 rows in set (0.05 sec)

mysql> select * from wp_options limit 1;
| option_id | option_name | option_value | autoload |
| 1 | siteurl | | yes |
1 row in set (0.01 sec)

mysql> exit

And that quickly the site was back up and functional, no restart required!

Hope that helps someone or me in the future.





Installing an rpm on Debian

My days of supporting Debian are numbered.  That isn’t a complaint, it will actually be nice to support one platform soon.  Until then I thought I’d share a little.

I needed to install oci8 to support a PHP application.  In doing this I ran into the following two problems, here are the solutions that worked for me (results may vary).

# apt-get install alien
# alien -d oracle-instantclient12.2-basic- 
error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch
error: cannot open Packages index using db5 - (-30969)
error: cannot open Packages database in /tmp/.rpmdb
error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch
error: cannot open Packages index using db5 - (-30969)
error: cannot open Packages database in /tmp/.rpmdb

The solution for this to do the following:

# rpm --rebuilddb
# alien -d oracle-instantclient12.2-basic-

That worked, next up the fix the pecl issue:

# pecl install oci8
Warning: Invalid argument supplied for foreach() in Command.php on line 249
PHP Warning: Invalid argument supplied for foreach() in /usr/share/php/PEAR/Command.php on line 249
PHP Stack trace:
PHP 1. {main}() /usr/share/php/peclcmd.php:0
PHP 2. require_once() /usr/share/php/peclcmd.php:31
PHP 3. PEAR_Command::getCommands() /usr/share/php/pearcmd.php:54
PHP 4. PEAR_Command::registerCommands() /usr/share/php/PEAR/Command.php:302

This one was not as obvious, it required a reinstallation as follows:

# apt-get purge php*-xml
# apt-get autoremove php*-xml
# apt-get install php-xml php7.0-xml
# apt-get purge php*-xml
# pecl install oci8
    (now add to the follwing ini files)
# vim /etc/php/7.0/apache2/php.ini 
# vim /etc/php/7.0/cli/php.ini 
# php --ri oci8

That should be it.  Good luck

md5sum differences on identical systems

In troubleshooting a problem a colleague noticed that there were md5sum differences in the files of identical systems:

ServerOne # md5sum /lib64/
27a605fdeaf7c835493a098213c9eec1  /lib64/

ServerTwo # md5sum /lib64/
13e3eb598abd09279efc05e215e77ae2 /lib64/libc-2.12.s

Analyzing a hex dump of the .so files showed cyclical differences at matching locations.  This is where I began helping to look at the problem.  After walking through the above (these files were the focus of an application error) I suggested we do a yum reinstall.  This resulted in different md5sums from those listed above.  The cyclical and matching locations are the link locations in the hex dump, as expected there are differences in the hex locations between systems.

After a little research I found the reason the md5sum’s don’t match is yum install uses prelink.  ‘prelink’ is a program that modifies ELF shared libraries and ELF dynamically linked binaries used by yum, therefore there would be differences in the md5sum even on identical systems.

To see the actual md5sum of the package before prelink is run, use the following command:

# prelink -y /lib64/ | md5sum
14486c78922a8dc8376a715d4c78d85a  -
 I ran this on the two systems in question and they match.  The files are the same on each systems, therefore this was a red hearing and the search continues.

hammer time

On a Satellite Server the Hammer command line management tool for provision hosts; editing the attributes; manipulating Satellite structures and mine for data. The annoying part of the hammer command is it (understandably) requires a username and password each time you issue the command. Something I would rather not expose in the bash history. So here is how to configure hammer so it doesn’t need to ask for the username and password.

Create the directory /root/.hammer (if you have used the command it should exist)

Create a file with a name “cli_config.yml” and put the following in the file:

 :host: 'https://satellite.server.url'
 :username: 'admin'
 :password: 'your-admin-password'

Make sure you set the permissions to secure this:

chmod 700 /root/.hammer
chmod 600 /root/.hammer/cli_config.yml

That is it!

Copying a file to multiple locations

I thought this was cool enough to share.  I have not had to use it however so I doubt it is all that useful 😉

xargs -n 1 cp -v foo.txt<<<"/tmp1/ /tmp2/ /tmp3/"

Here we are copying one file named foo.txt to multiple directories called /tmp/1/, /tmp/2/, and /tmp/3 using xargs.  The xargs command construct argument list(s) and execute utility such as cp.

If you find a use for this let me know.  Backup copies seems useful but other than that…?