Friday, June 27, 2014

RPM Command: 15 Examples to Install, Uninstall, Upgrade, Query RPM Packages

RPM command is used for installing, uninstalling, upgrading, querying, listing, and checking RPM packages on your Linux system.
RPM stands for Red Hat Package Manager.
With root privilege, you can use the rpm command with appropriate options to manage the RPM software packages.

In this article, let us review 15 practical examples of rpm command.
Let us take an rpm of Mysql Client and run through all our examples.

1. Installing a RPM package Using rpm -ivh

RPM filename has packagename, version, release and architecture name.
For example, In the MySQL-client-3.23.57-1.i386.rpm file:
  • MySQL-client – Package Name
  • 3.23.57 – Version
  • 1 – Release
  • i386 – Architecture
When you install a RPM, it checks whether your system is suitable for the software the RPM package contains, figures out where to install the files located inside the rpm package, installs them on your system, and adds that piece of software into its database of installed RPM packages.
The following rpm command installs Mysql client package.
# rpm -ivh  MySQL-client-3.23.57-1.i386.rpm
Preparing...                ########################################### [100%]
   1:MySQL-client           ########################################### [100%]
rpm command and options
  • -i : install a package
  • -v : verbose
  • -h : print hash marks as the package archive is unpacked.
You can also use dpkg on Debianpkgadd on Solarisdepot on HP-UX to install packages.

2. Query all the RPM Packages using rpm -qa

You can use rpm command to query all the packages installed in your system.
# rpm -qa
cdrecord-2.01-10.7.el5
bluez-libs-3.7-1.1
setarch-2.0-1.1
.
.
  • -q query operation
  • -a queries all installed packages
To identify whether a particular rpm package is installed on your system, combine rpm and grep command as shown below. Following command checks whether cdrecord package is installed on your system.
# rpm -qa | grep 'cdrecord'

3. Query a Particular RPM Package using rpm -q

The above example lists all currently installed package. After installation of a package to check the installation, you can query a particular package and verify as shown below.
# rpm -q MySQL-client
MySQL-client-3.23.57-1

# rpm -q MySQL
package MySQL is not installed
Note: To query a package, you should specify the exact package name. If the package name is incorrect, then rpm command will report that the package is not installed.

4. Query RPM Packages in a various format using rpm –queryformat

Rpm command provides an option –queryformat, which allows you to give the header tag names, to list the packages. Enclose the header tag with in {}.
# rpm -qa --queryformat '%{name-%{version}-%{release} %{size}\n'
cdrecord-2.01-10.7 12324
bluez-libs-3.7-1.1 5634
setarch-2.0-1.1 235563
.
.

#

5. Which RPM package does a file belong to? – Use rpm -qf

Let us say, you have list of files and you would want to know which package owns all these files. rpm command has options to achieve this.
The following example shows that /usr/bin/mysqlaccess file is part of the MySQL-client-3.23.57-1 rpm.
# rpm -qf /usr/bin/mysqlaccess
MySQL-client-3.23.57-1
  • -f : file name

6. Locate documentation of a package that owns file using rpm -qdf

Use the following to know the list of documentations, for a package that owns a file. The following command, gives the location of all the manual pages related to mysql package.
# rpm -qdf /usr/bin/mysqlaccess
/usr/share/man/man1/mysql.1.gz
/usr/share/man/man1/mysqlaccess.1.gz
/usr/share/man/man1/mysqladmin.1.gz
/usr/share/man/man1/mysqldump.1.gz
/usr/share/man/man1/mysqlshow.1.gz
  • -d : refers documentation.

7. Information about Installed RPM Package using rpm -qi

rpm command provides a lot of information about an installed pacakge using rpm -qi as shown below:
# rpm -qi MySQL-client
Name        : MySQL-client                 Relocations: (not relocatable)
Version     : 3.23.57                           Vendor: MySQL AB
Release     : 1                             Build Date: Mon 09 Jun 2003 11:08:28 PM CEST
Install Date: Mon 06 Feb 2010 03:19:16 AM PST               Build Host: build.mysql.com
Group       : Applications/Databases        Source RPM: MySQL-3.23.57-1.src.rpm
Size        : 5305109                          License: GPL / LGPL
Signature   : (none)
Packager    : Lenz Grimmer
URL         : http://www.mysql.com/
Summary     : MySQL - Client
Description : This package contains the standard MySQL clients.
If you have an RPM file that you would like to install, but want to know more information about it before installing, you can do the following:
# rpm -qip MySQL-client-3.23.57-1.i386.rpm
Name        : MySQL-client                 Relocations: (not relocatable)
Version     : 3.23.57                           Vendor: MySQL AB
Release     : 1                             Build Date: Mon 09 Jun 2003 11:08:28 PM CEST
Install Date: (not installed)               Build Host: build.mysql.com
Group       : Applications/Databases        Source RPM: MySQL-3.23.57-1.src.rpm
Size        : 5305109                          License: GPL / LGPL
Signature   : (none)
Packager    : Lenz Grimmer
URL         : http://www.mysql.com/
Summary     : MySQL - Client
Description : This package contains the standard MySQL clients.
  • -i : view information about an rpm
  • -p : specify a package name

8. List all the Files in a Package using rpm -qlp

To list the content of a RPM package, use the following command, which will list out the files without extracting into the local directory folder.
$ rpm -qlp ovpc-2.1.10.rpm
/usr/bin/mysqlaccess
/usr/bin/mysqldata
/usr/bin/mysqlperm
.
.
/usr/bin/mysqladmin
  • q : query the rpm file
  • l : list the files in the package
  • p : specify the package name
You can also extract files from RPM package using rpm2cpio as we discussed earlier.

9. List the Dependency Packages using rpm -qRP

To view the list of packages on which this package depends,
# rpm -qRp MySQL-client-3.23.57-1.i386.rpm
/bin/sh
/usr/bin/perl

10. Find out the state of files in a package using rpm -qsp

The following command is to find state (installed, replaced or normal) for all the files in a RPM package.
# rpm -qsp MySQL-client-3.23.57-1.i386.rpm
normal        /usr/bin/msql2mysql
normal        /usr/bin/mysql
normal        /usr/bin/mysql_find_rows
normal        /usr/bin/mysqlaccess
normal        /usr/bin/mysqladmin
normal        /usr/bin/mysqlbinlog
normal        /usr/bin/mysqlcheck
normal        /usr/bin/mysqldump
normal        /usr/bin/mysqlimport
normal        /usr/bin/mysqlshow
normal        /usr/share/man/man1/mysql.1.gz
normal        /usr/share/man/man1/mysqlaccess.1.gz
normal        /usr/share/man/man1/mysqladmin.1.gz
normal        /usr/share/man/man1/mysqldump.1.gz
normal        /usr/share/man/man1/mysqlshow.1.gz

11. Verify a Particular RPM Package using rpm -Vp

Verifying a package compares information about the installed files in the package with information about the files taken from the package metadata stored in the rpm database. In the following command, -V is for verification and -p option is used to specify a package name to verify.
# rpm -Vp MySQL-client-3.23.57-1.i386.rpm
S.5....T c        /usr/bin/msql2mysql
S.5....T c        /usr/bin/mysql
S.5....T c        /usr/bin/mysql_find_rows
S.5....T c        /usr/bin/mysqlaccess
The character in the above output denotes the following:
  • S file Size differs
  • M Mode differs (includes permissions and file type)
  • 5 MD5 sum differs
  • D Device major/minor number mismatch
  • L readlink(2) path mismatch
  • U User ownership differs
  • G Group ownership differs
  • T mTime differs

12. Verify a Package Owning file using rpm -Vf

The following command verify the package which owns the given filename.
# rpm -Vf /usr/bin/mysqlaccess
S.5....T c /usr/bin/mysql
#

13. Upgrading a RPM Package using rpm -Uvh

Upgrading a package is similar to installing one, but RPM automatically un-installs existing versions of the package before installing the new one. If an old version of the package is not found, the upgrade option will still install it.
# rpm -Uvh MySQL-client-3.23.57-1.i386.rpm
Preparing...    ########################################### [100%]
1:MySQL-client          ###########################################

14. Uninstalling a RPM Package using rpm -e

To remove an installed rpm package using -e as shown below. After uninstallation, you can query using rpm -qa and verify the uninstallation.
# rpm -ev MySQL-client

15. Verifying all the RPM Packages using rpm -Va

The following command verifies all the installed packages.
# rpm -Va
S.5....T c /etc/issue
S.5....T c /etc/issue.net
S.5....T c /var/service/imap/ssl/seed
S.5....T c /home/httpd/html/horde/ingo/config/backends.php
.
.
S.5....T c /home/httpd/html/horde/ingo/config/prefs.php
S.5....T c /etc/printcap
Link:http://www.thegeekstuff.com/2010/07/rpm-command-examples/

Monday, June 9, 2014

Daemon Showdown: Upstart vs. Runit vs. Systemd vs. Circus vs. God

We write a lot of daemons : programs which run on servers in the background, like an HTTP server, or a database. Once we've written the programs, though, we have to run them, and running programs as daemons is surprisingly heavy on details; it's fraught with perils for the unwary. If you go the traditional Unix way, you do some magic with double-forking and pid files and init scripts, and it's horribly tedious. If you run in a detached session of screen or tmux , you end up doing everything manually. In both cases, there's nothing in place to restart your program if it crashes. Steve Huffman, one of Reddit's founders, will tell you from experience just how bad an idea this is:
In the early days of Reddit, we didn’t really have any crash protection. We had tons of errors, and Reddit would occasionally lock up, freeze, or get in an infinite loop, any number of ways of bringing down the site. I used to have to sleep with my laptop and I would wake up every couple of hours and see if Reddit was working, and restart it. It was the worst feeling in the world; totally mentally draining.
To reclaim their ability to sleep, they had the computer keep watch for them, resurrecting dead processes. This also let them automate health checks: when Reddit's front page became unresponsive, they could detect this automatically and simply kill the offending process, secure in the knowledge that the supervisor program would quickly bring it back up again. They could sleep right through this fleeting death and rebirth, and only a few people would notice.
There are some good tools for keeping programs running as daemons, and all of them make it easy for you. If you're on Ubuntu, Upstart is the default, and it's a good choice. If you want something less monolithic, Runit is featureful and appealingly Unixy. Several Linux distributions use systemd as their init, and for our purposes it's comparable to Upstart. Other interesting options include Supervisord , God , andCircus . Unfortunately there is no clear best answer here, so which one you want depends on your needs. Let's compare.

Upstart

Used as the default init replacement on Ubuntu, Upstart is very good at starting things up. The Upstart Manual can be kind of overwhelming, but the config files themselves are pretty simple. For example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# myserver - it's my server
#
# myserver listens on a port and, like, does things. Great things.

description     "My cool server"

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 10 5

setuid myuser
exec /usr/local/bin/myserver --port 8080
Let's walk through this. The comments at the top customarily include a one-line description and an optional longer description below – a lot like a git commit message. The description line is, again, for your own use. Thestart on runlevel [2345] part means to start the service when the machine is started. The respawn and respawn limit commands tell Upstart to restart the server if it dies, but stop trying to restart it after 10 failures in 5 seconds. Finally, Upstart sets the uid to myuser and runs the server. Make sure to setuid , because if you don't include it, your server will be run as root by default. This is a profoundly weird default – it's usually not what you want, and it's very dangerous. So, be careful with this.
Upstart expects myserver not to fork. Any output from myserver on stdout or stderr gets sent, by default, to a log file in /var/log/upstart/ . Both of these things can be changed; Upstart can handle the classic forking style of daemons, and you can plug in other options for logging.
Upstart is one of the best options for running traditional forking daemons, since it actually uses ptrace to make sure it's watching the correct process ids. The system for this is very slick.

Runit

Upstart, while admirably simple to use, does have some shortcomings. Its default logging, for example, does not support log rotation and is only compatible withlogrotate if you use the copytruncate hack. Upstart's options for running servers sometimes don't include all that you might want; for instance, it currently has no way to adjust the maximum number of file descriptors, or limit memory usage. Runit is an interesting alternative, inspired by D. J. Bernstein's daemontools : it's a collection of programs which each do one thing, and together, they have a lot of features that Upstart lacks. If you want a gentle introduction to using Runit, the blog post Runit for Ruby (And Everything Else) is very well-written.
The two best things about Runit are the chpst and svlogd programs. With chpstyou can start a program with a ch anged p rocess st ate:
chpst -u myuser -/ /var/roots/mychroot -o 10000 -n 3 /usr/local/bin/myserver --port 8080
This runs the command /usr/local/bin/myserver --port 8080 as usermyuser , chrooted to /var/roots/mychroot , with a maximum of 10000 open file descriptors, at a niceness level of 3. Some of these options are available with Upstart, and some aren't. The beauty of chpst , though, is that you can use it with anything. You can use chpst with Upstart, or Supervisord, or whatever else strikes your fancy. And because Runit relies entirely on chpst for this sort of thing, you end up with completely reproducible execution environments: if you run the Runit script for a service by hand, you get the same environment as you would get if Runit had run it. This lack of magic is invaluable when debugging.
Similar to chpst , Runit's svlogd can be used with anything to provide log-handling. It reads text-based logs on stdin, and either writes that to rotated log files, or sends it to syslog, or some combination thereof.
Runit is a decent way to run daemons, and it has useful pieces that work with just about anything. We use it for most of our services. The one caveat to be aware of is that Runit expects daemons not to fork; this is great for servers that you write yourself, but it can be inconvenient if you try to use Runit with some other programs out there.

systemd

The always-lowercase systemd is used as the default init on a fair number of Linuxes, most notably the Redhat/Fedora family of distributions. If you're using one of these and would like to go with the already-installed option, this blog post is a good introduction . Since it's pretty similar to Upstart, the easiest way to describe it is to talk about some of the differences. First, obviously, the configuration files have a different syntax, but that's more of a superficial difference – the concepts are the same. More noteworthy are the ways that systemd handles logging, resource limiting, and daemons that fork.
Systemd is serious about logging (though I suspect they may have over-engineered it a bit). By default, all processes have their stdout and stderr connected to systemd's logging facility (described here and compared with syslog here ), but it can also go to syslog, the kernel log buffer, a socket, a tty, or any combination of these. Docs for the logging are here .
Systemd has a very clever solution to the problem of tracking daemons that fork, which coincidentally happens to handle resource limiting at the same time. Where Upstart uses ptrace to watch the forking, systemd runs each daemon in a control group (requires Linux 2.6.24 or newer) from which it can not escape with any amount of forking. This allows easy resource limiting , both for forking and non-forking daemons, since control groups were made for this sort of thing.
More documentation can be found on the systemd web site . Or, for a reference,the man pages are here .

Other contenders: Supervisord, God, Circus

Supervisord is pretty convenient to set up, and seems to get a fair amount of use in Pythonland. Its main advantage over Upstart, Runit, and systemd is that Supervisord can easily start pools of several instances of a program. This is useful if you want a pool of worker processes, e.g. as FastCGI workers. Supervisord expects programs to not daemonize, though it can handle daemonized programs with a pid file workaround. This workaround does not handle edge cases as well as Upstart or systemd, but it should work fine most of the time.
The hard-to-google God hails from the Ruby world. Its great virtue is programmability: you decide what programs to start, how to run them, etc., and God will monitor them and keep them up and running. Suppose you have three back-end web servers, which you want to run on three different ports, so that a reverse proxy can load-balance across all of them. The config file is just a Ruby program, so a simple loop can bring up all three servers. The God web site has examples with code. If you're comfortable writing Ruby and you have interesting needs, God might be what you're looking for.
Mozilla's Circus is perhaps the most interesting of the three. It's programmable in Python, it can handle pools of workers, and it can bind sockets for you . This last point is especially neat: Circus can act as an HTTP server container by binding to a socket, starting up a pool of worker processes, and passing them the file descriptor of the socket. It acts a lot like Gunicorn , but more powerful. Circus also handles logging and log rotation, has a web-based console for watching the supervised processes, and sends a stream of all events over a ZeroMQ PUB socket . It's more actively developed than Supervisord or God, but already stable enough for production – and it has the best docs. Here are some examples of how to use it.If I wanted to serve up a web site with a WSGI stack, Circus is the option I would look at first.
Any of these three can be run under, say, Upstart. They don't want to be an init replacement, so they play nice with everything else.

Pick something

Hopefully this infodump is a useful comparison, but the most important thing isn'twhich of these options you use; what's important is that you use something.Automating server administration a bit more is almost always a win. The more things you can solve once and then stop thinking about, the better.
It's also worth mentioning that a simple crash is far from the only reason why a daemon might need to be restarted. Several of our server health checks can kill unresponsive processes, if something has frozen up or gone into a weird state. This can also be a temporary stopgap against hard-to-debug intermittent memory leaks: if the server is using too much memory, kill it! We have an ever-growing number of health checks keeping our servers in working order, because it's easy to add more. We're not quite writing crash-only software , but it turns out that you can get a lot of its advantages by making crashes cheap.