MySQL DBA - Tips and Techniques Magazine

23 Oct 2014

InnoDB: ERROR: the age of the last checkpoint

We just had this error message from "mk-lluprequal-1"

 

 

080331 10:57:46  InnoDB: ERROR: the age of the last checkpoint is

9433713,

InnoDB: which exceeds the log group capacity 9433498.

InnoDB: If you are using big BLOB or TEXT rows, you must set the

InnoDB: combined size of log files at least 10 times bigger than the

InnoDB: largest such row.

 

 

 

Can you please explain why someone is running alter table command on a

production system.  There is a transaction that may be too big for the

allocated log size.  Currently its set to 5m.

 

| 33927407 | prequal_rw | dhcp-26-55.hk.uk.intranet:39713   | prequal |

Query       | 434     | copy to tmp table

| alter table LineTest add key TestDate(TestDate) |

 

 

 

 

http://crazytoon.com/2007/08/16/mysql-innodb-error-the-age-of-the-last-checkpoint-is-number/

Install MYSQL on LINUX

Here is an example of doing the install for PAT environment.


 

1. Define requirements:

 

Install on mk-pat-simulator-1 , 10.44.42.2

 

Install latest mySQL version 5.0

 

Version of linux we have is 

 

view /etc/redhat-release

 

 

CentOS release 5 (Final)

 

 

The processor type is

 

uname -a

 

Linux mk-pat-simulator-1.uk.intranet 2.6.18-8.el5 #1 SMP Thu Mar 15 19:46:53 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

 

 

Ensure you have root access .

 

 

2.  Follow instructions on

 

2.4.9. Installing MySQL from RPM Packages on Linux

 

 

( Takes you to the community download page

 

Choose the correct Linux version and processor.

 

Red Hat Enterprise Linux 5 RPM (x86) downloads  )

 

Download Server and Client.

 

Saved to desktop & Upload to the server (due to security restrictions on server).

 

 

1.     Do minimal install

 

sudo bash

 

cd /home/hlakhan

 

Server:

 

rpm –ivh MySQL-server-community-5.0.51a-0.rhel5.i386.rpm

 

Client:

 

rpm -ivh MySQL-client-community-5.0.51a-0.rhel5.i386.rpm

 

 

 

[

 NB  Server gave error messages because an older version was already installed

 

rpm -ivh MySQL-client-community-5.0.51a-0.rhel5.i386.rpm

Preparing...                ########################################### [100%]

        file /usr/bin/msql2mysql from install of MySQL-client-community-5.0.51a-0.rhel5 conflicts with file from package mysql-5.0.22-2.1

 

 

Can check to see which mysql packages are already installed:

 

rpm -qa |grep -i mysql

 

mysql-5.0.22-2.1

mysql-5.0.22-2.1

 

 

Need to erase the old ones :

 

rpm -e --allmatches mysql-5.0.22-2.1

 

           

And then install – works!

]

           

 

To check the install :

 

 

mysql

 

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 1

Server version: 5.0.51a-community MySQL Community Edition (GPL)

 

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

 

mysql> exit

 

 

ls -lrt  /var/lib/mysql

 

/sbin/service mysql restart

 

Shutting down MySQL.[  OK  ]

Starting MySQL[  OK  ]

 

 

To set the passwords

 

2.4.16.3. Securing the Initial MySQL Accounts

 

20 Oct 2014

Remove user from master but allow it read only access on slave

A user "crystal" existed on Master-1 and Slave-2. To prevent load on the master, limit crystal users access to slave only.

Make sure you know the password for the user you are removing, as you will need to recreate it on the slave

 

Log on to mysql on master as root

 

Use mysql;

delete from user where user='crystal';

flush privileges;

 

Log off root account, and try and connect as user to make sure account has been deleted successfully

 

mysql -ucrystal -p

Enter password:

ERROR 1045 (28000): Access denied for user 'crystal'@'localhost' (using password: YES)

 

Deleting the user will replicate to the slave, so need to recreate the user on the slave.

 

Confirm that the account has been deleted from the slave

 

mysql -ucrystal -p

Enter password:

ERROR 1045 (28000): Access denied for user 'crystal'@'localhost' (using password: YES)

 

Recreate the account on the slave, as root

 

            Use mysql;


            grant select on *.* to 'crystal'@'%' identified by '<password>';


            flush privileges;


            select * from user;



Log off root account, try to connect using new account and check access

           

            mysql -ucrystal –p


            use CALLDRIVER;

            select * from Calls limit 1;



OK.


16 Oct 2014

mysqldump Backup script using Bzip

Here is an example of a backup script using bzip.  This takes longer but compresses upto 20% more.

 

 

backup_dbs_bzip.sh

 

#!/bin/sh

 

EMAIL_LIST="dbateam@uk.abc.com"

 

date=`/bin/date +"%d-%m-%y-%H:%M"`

umask 066

( /usr/bin/mysqldump -uroot --single-transaction --all-databases | bzip2 -cz1) > /home/backup/all-$date-`hostname -s`.sql.bz2 2> /home/backup/all-$date-`hostname -s`.sql.err.log

 

ls -ltr /home/backup/all-$date-`hostname -s`.sql.bz2 > /home/backup/filelist.out

ls -ltr /home/backup/all-$date-`hostname -s`.sql.err.log >> /home/backup/filelist.out

 

mail -s "mySQL Backup on `hostname -s`" $EMAIL_LIST < /home/backup/filelist.out

 

# Clear up old data, anything older then 20 days

find /home/backup/*.bz2 -ctime +20 -exec rm {} \;

find /home/backup/*.log -ctime +20 -exec rm {} \;

 

 

 

backup_dbs_gzip.sh

 

#!/bin/sh

 

EMAIL_LIST="dbateam@uk.abc.com"

 

date=`/bin/date +"%d-%m-%y-%H:%M"`

umask 066

( /usr/bin/mysqldump -uroot --single-transaction --all-databases | gzip) > /home/backup/all-$date-`hostname -s`.sql.gz 2> /home/backup/all-$date-`hostname -s`.sql.err.log

 

ls -ltr /home/backup/all-$date-`hostname -s`.sql.gz > /home/backup/filelist.out

ls -ltr /home/backup/all-$date-`hostname -s`.sql.err.log >> /home/backup/filelist.out

 

mail -s "mySQL Backup on `hostname -s`" $EMAIL_LIST < /home/backup/filelist.out

 

# Clear up old data, anything older then 20 days

find /home/backup/*.gz -ctime +20 -exec rm {} \;

find /home/backup/*.log -ctime +20 -exec rm {} \;

 

 

 

for multiple databases, use  --database db_name1 db_name2  , instead of –all-databases

 

Replication Failure Due to Duplicates

1.     Log onto the slave and check slave status

 

2.     If the error is due to a duplicate, see example below, you can first set skip counter to see if it skips it, if not then just delete the row on the slave and restart replication.  If there are many duplicates, then use the script in part c).

 

 

a) before skipping, just check if it is the exact same row being reinserted that fails,

select * from <table> where  <key field> = <see value in message>,    If all the returned columns match what was being inserted it means the master has sent the records twice.  This is a known intermittant bug in mySQL that was supposed to be fixed in vn 4, but still happens in vn 4 and 5.  The work around is to put the following entry into the my.cnf on the slave to automatically skip these duplicates.

 

[mysqld]

slave-skip-errors = 1062

 

..and restart the server.

 

b) try skipping one row manually as in this example:

 

 

ERROR :

Subject: mySQL error on mk-lluprequal-2

 

080130 11:57:39 [ERROR] Slave: Error 'Duplicate entry '7851915' for key 1' on query. Default database: 'prequal'. Query: 'insert into checker_requests (api_version, package, cli, ip, ts) values ('1.7','btv9','01793762637','195.224.207.200',now())', Error_code: 1062 080130 11:57:39 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.000042' position 771163277

 

Steps taken :

 

Show slave status \G

Stop slave;

set global sql_slave_skip_counter=1;

start slave;

Show slave status \G

 

and repeat until Show slave status \G shows that there are no more errors.  Also move the error log.

If the error is still there, then try deleting it as follows:

 

b) try skipping as in this example:

 

 

      mysql> show slave status \G

*************************** 1. row ***************************

             Slave_IO_State: Waiting for master to send event

                Master_Host: mk-fusion-1.uk.intranet

                Master_User: repl

                Master_Port: 3306

              Connect_Retry: 60

            Master_Log_File: mysql-bin.000038

        Read_Master_Log_Pos: 87465725

             Relay_Log_File: mk-fusion-2-relay-bin.000002

              Relay_Log_Pos: 84940721

      Relay_Master_Log_File: mysql-bin.000038

           Slave_IO_Running: Yes

          Slave_SQL_Running: No

            Replicate_Do_DB:

        Replicate_Ignore_DB:

         Replicate_Do_Table:

     Replicate_Ignore_Table:

    Replicate_Wild_Do_Table:

Replicate_Wild_Ignore_Table:

                 Last_Errno: 1062

                 Last_Error: Error 'Duplicate entry 'oxythony77@tiscali.co.uk' for key 1' on query. Default database: 'WALLEDGARDEN'. Query: 'insert into USERS(`DIALLEDNUMBER`,`USERNAME`,`DATE`,`SERVICEID`)  values('NONE','oxythony77@tiscali.co.uk','2008-01-17 17:26:19','2')'

               Skip_Counter: 0

        Exec_Master_Log_Pos: 84983826

            Relay_Log_Space: 87426970

            Until_Condition: None

             Until_Log_File:

              Until_Log_Pos: 0

         Master_SSL_Allowed: No

         Master_SSL_CA_File:

         Master_SSL_CA_Path:

            Master_SSL_Cert:

          Master_SSL_Cipher:

             Master_SSL_Key:

      Seconds_Behind_Master: NULL

 

Using this example, issue the following commands

·       use WALLEDGARDEN;

·       delete  from USERS where username = 'oxythony77@tiscali.co.uk';          

·       stop slave;                                                                                          

·       start slave;                                                                                           

·        show slave status \G                                                                    

 

            If you get more than 1 duplicate, repeat the delete with the new value and stop / restart the slave again. Continue until show slave status shows no more duplicates

 

 

c) Script to remove many duplicates

 

on msh/bth where sockets are used:

 

#!/bin/sh

 

Slave_SQL_Running=`/slave/mysql-bin/mysql/bin/mysql -uroot -p*** -S /tmp/mysql-slave.sock -e "SHOW SLAVE STATUS\G" |  grep -i Slave_SQL_Running | awk '{ p

rint $2 }'`

 

echo "SLAVE running-->$Slave_SQL_Running"

 

if [ $Slave_SQL_Running != 'Yes' ]

then

        echo "mySQL not running.  Stopping slave...."

        /slave/mysql-bin/mysql/bin/mysql -uroot -p***-S /tmp/mysql-slave.sock -e "STOP SLAVE";

        /slave/mysql-bin/mysql/bin/mysql -uroot -p***-S /tmp/mysql-slave.sock -e "set global sql_slave_skip_counter=1";

        /slave/mysql-bin/mysql/bin/mysql -uroot -p***-S /tmp/mysql-slave.sock -e "START SLAVE";

else

        echo "SLAVE running-->$Slave_SQL_Running"

 

fi

 

 

On most of the other databases:

 

[root@mk-nn-radproxy-5 ~]# vi skip_duplicate.sh

 

#!/bin/sh

 

export PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/home/hlakhan/bin

 

Slave_SQL_Running=`mysql -uroot -p'Bisohh!b' -e "SHOW SLAVE STATUS\G" |  grep -i Slave_SQL_Running | awk '{ print $2 }'`

 

echo "SLAVE running-->$Slave_SQL_Running"

 

if [ $Slave_SQL_Running != 'Yes' ]

then

        echo "mySQL not running.  Stopping slave...."

        mysql -uroot --p*** -e "STOP SLAVE";

        mysql -uroot --p*** -e "set global sql_slave_skip_counter=1";

        mysql -uroot --p*** -e "START SLAVE";

else

        echo "SLAVE running-->$Slave_SQL_Running"

 

fi

 

14 Oct 2014

Setup replication user “REPL”

1) Add the following master information to the new servers my.cnf file.  This was taken from the master servers my.cnf file.


# The replication master for this slave - required
master-host     =   mk-myserver-1.uk.intranet
#
# The username the slave will use for authentication when connecting
# to the master - required
master-user     =   repl
#
# The password the slave will authenticate with when connecting to
# the master - required
master-password =   lkhhdsgs7


2) Check to see if new slave can connect to master using user "repl".

Run on slave
 - mysql -hmk-myserver-1.uk.intranet -urepl -plkhhdsgs7

 If not,  then run the following on master :

GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 
repl@"mk-myserver-3.uk.intranet" IDENTIFIED BY 'lkj5lv327'

check again to see if slave can connect to master :

run on slave
- mysql -hmk-myserver-1.uk.intranet -urepl –p<paswd>

You should be able to connect now.


Restoring a backup to new server

0.     Connect to New server


1.     Ensure mysql is up and running

 

-        ps auw | grep mysql

-        ps –ef |grep mysql (LINUX)

 

2.     Connect to mysql to make sure you can.

 

-        mysql

-        mysql –uroot -pXXXXX

 

3.     Run Restore

 

-        bunzip2 < MYSQL-BACKUP-FILENAME.bz2 | mysql -uroot –pXXXXXX

 

or

 

-        gunzip < MYSQL-BACKUP-FILENAME.gz | mysql -uroot –pXXXXXX

 

Take Static Backup of MYSQL Database

To take a static backup of a MYSQL database:

 

NOTE :

Backup example

 

mysqldump -uroot -pPASSWORD --single-transaction --all-databases | gzip) >

MYSQL-BACKUP-FILENAME.gz 2> MYSQL-BACKUP-ERROR.log

 

 

 

 

1) Stop Apache (Systems Team)

 

SYSTEMS WILL DO THIS

 

 

2) Stop slave on mk-lluprequal-2 (DBA)

 

Connect to mysql

mysql -uroot -p'XXXXXXX'

 

 

type in

 

mysql > show slave status \G

 

mysql > stop slave

 

mysql > show slave status \G

"Keep information safe somewhere"

 

 

3) Take Backup of MYSQL DB (DBA)

 

(check crontab for backup script location)

/home/backup/backup_dbs.sh

 

 

4) Take note of replication log information.

 

Connect to mysql

mysql -uroot -p'XXXXXXX'

 

 

type in

 

duplicate command from step 3

mysql > show slave status \G

 

"Keep information safe somewhere"

 

 

5) Restart slave on mk-lluprequal-2 (DBA)

 

 

mysql > start slave \G

 

 

6) Restart Apache (Systems Team)

 

SYSTEMS WILL DO THIS

Install MYSQL on FREEBSD

1)     DOWNLOAD required MYSQL Version on Free BSD

 

You can try getting the server to download the software, however, it will install the software in /usr/local/bin

 

            cd /usr/ports/databases/

cd mysql50-server                  (or the directory relevant to the version of mysql you wish to install)

make install clean

 

To start the database

 

            vi /etc/rc.conf --> mysql_enable="YES"

 

            /usr/local/etc/rc.d/mysql-server start

 

            You can create a my.cnf if required, otherwise it will use all the defaults

 

The server will attempt to connect to various ftp sites to get the software, and hopefully will download it for you. 

 

 

Alternatively

 

2)     DOWNLOAD required MYSQL Version on Free BSD and FTP/SCP to new box

 

You can use pkg_add to install mySQL but I prefer to do the following steps (it's a more complicated process on Free BSD):-

Add mySQL user and group

 

  • pw groupadd mysql
  • pw useradd mysql
  • pw groupmod mysql -M mysql
  • pw groupshow mysql

 

Create directory where mySQL resides

 

  • mkdir /opt/tiscali/mysql_5.0.19

 

Uncompress and untar ball file

 

In this case the file contains the server and client mySQL binaries. Run as ROOT.

 

·       shell> cd /opt/tiscali/mysql_5.0.19

·       shell> gunzip < /path/to/mysql-VERSION-OS.tar.gz | tar xvf –

·       shell> ln -s full-path-to-mysql-VERSION-OS mysql

·       shell> cd mysql

·       shell> chown -R mysql .

·       shell> chgrp -R mysql .

·       shell> scripts/mysql_install_db --user=mysql

·       shell> chown -R root .

·       shell> chown -R mysql data

·       shell> bin/mysqld_safe --user=mysql &

·       shell> ln -s /opt/tiscali/mysql_5.0.19/mysql    /usr/local/mysql

·       shell> cp /opt/tiscali/mysql_5.0.19/mysql-standard-5.0.19-freebsd5.3-i386/support-files/mysql.server /etc/rc.d/mysql.server

·       shell> cp /opt/tiscali/mysql_5.0.19/mysql-standard-5.0.19-freebsd5.3-i386/support-files/mysql.server /usr/local/etc/rc.d/mysql-server.sh

·       shell> vi /etc/rc.conf --> mysql_enable="YES"

·       shell> ln -s /etc/my.cnf      /opt/tiscali/mysql_5.0.19/mysql/data/my.cnf

 

If the tar files that you install include MySQL-server, the mysqld server should be up and running after installation. You should be able to start using MySQL.

 

Move Master back to Original Server


After losing the Master and having promoted a slave to Master.  Now Move back to the Master.



On Old Master Database

·       Log on to my sql and make a note of Master_log_file and read_master_log_pos

 

            mysql –uuser-ppassword

 

            Show master status

 

+---------------------+-----------+-------------------+-----------------------+

| File                        | Position | Binlog_Do_DB | Binlog_Ignore_DB |

+---------------------+-----------+-------------------+------------------------+

| mysql-bin.000007 |          98 |                            |                                |

+---------------------+-----------+--------------------+-----------------------+

 

1.     Shut down master database

           

            /sbin/service mysql stop        or         /etc/rc.d/mysql.server stop

 

·       Take a backup of the database

 

·       Copy backup file to PC

 

            cd <backup_file_location>

 

            cp <backup_file> /tmp

 

            chmod 777 <backup_file>

 

            scp file to PC

 

·       DO NOT START DATABASE

 

                             

On New Master Database

·       Install mysql

 

·       Copy backup file to slave server – make sure there is enough space!

 

·       Check you can connect to the database

 

            Mysql –uroot –ppassword

 

·       Restore backup file to database (assumes bzip'd file)

 

            cd <file_name>

 

            nohup bunzip2 <file_name> | mysql –uroot –ppassword &

 

·       Check progress – are files being created

 

            cd  /opt/Tiscali/mysql.5.0.19             or         cd /var/lib/mysql

 

            ls –ltr

 

            cd data            or         cd ibdata

 

            ls –ltr

 

·       View my.cnf, check

 

            server_id = a unique value

 

            all lines starting master- are commented out

 

            remove 'read only' or 'readonly' entry

 

 

On Old Master – make this the slave

·       Start mysql

 

·       restart mysql as a test:

 

            sudo bash

 

            /sbin/service mysql stop;

            /sbin/service mysql start;

 

 

·       Check slave status

 

            mysql –uroot –ppassword 

 

            show slave status;

 

(Check replication is ok)

 

 


MyISAM Table Corruption ( Check and Fix )

1)     Get enough information from your error  email(server name and corrupted table)

2)     login as yourself (using keys) – e.g. mraj

3)     sudo bash ( become root user)

4)     Check Table for corruption :

 

4a) From Command line(logged in as root)

- mysqlcheck –uroot –p[password] [databasename] tablename

 

            or

 

4b) From inside MYSQL

-        mysql -uroot –p[password]

-        use [databasename];

-        check table [tablename];

 

e.g. should give you similar output

+--------------+-------+----------+----------------------------------------------------------+

| Table        | Op    | Msg_type | Msg_text                                                 |

+--------------+-------+----------+----------------------------------------------------------+

| ebms.message | check | warning  | Table is marked as crashed                               |

| ebms.message | check | error    | Found 3138852 keys of 3138848                            |

| ebms.message | check | error    | Corrupt                                                  |

+--------------+-------+----------+----------------------------------------------------------+

 

 

5)     Repair Table

 

Using MYSQL to repair the above table called 'message':

 

-        mysql -uroot –p[password]

-        use [databasename];

-        mysql> repair table message

 

+--------------+--------+----------+----------+

| Table        | Op     | Msg_type | Msg_text |

+--------------+--------+----------+----------+

| ebms.message | repair | status   | OK       |

+--------------+--------+----------+----------+

1 row in set (1 min 14.83 sec)

 

6)     Check Table again for corruption, follow step 4a or 4b.


 


12 Oct 2014

Move error log to stop error log messages

Once an error has been resolved, to stop it sending email alerts we need to clear the logfile down.

 

a) find the log

 

ps -ef|grep mysql

 

root     12201     1  0 Jan17 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/mk-fusion-2.uk.intranet.pid

 

b) clear the logfile down

 

cd /var/lib/mysql

 

cp mk-fusion-2.uk.intranet.err mk-fusion-2.uk.intranet.err.040208

 

> mk-fusion-2.uk.intranet.err

 

c) check permissions and owner of logfile are still correct

 

ls –lrt mk-fusion-2.uk.intranet.err

 

-rw-rw-rw-  1 mysql mysql         0 Feb  4 09:36 mk-fusion-2.uk.intranet.err

 

6 Oct 2014

Use Streams, Pipes, Redirects, Grep and Cut!

BASIC  - Cat, Split, Diff

 

               cat - concatenate files and display

               wc -l      (line count)

cat file* | wc -l   (count of total line in file*)

split -l 10 file2      (splits file2 into 10 lines each in files named xaa , xab )

diff xaa xab   (shows difference between two files)

 

STREAMS (std out, stderr, stdin) & REDIRECTS

cat file* > newfile.txt        (  > redirect of stdout)

echo "new entry" >> newfile.txt   (appends)

 

ls missingfile 2>  error.log   ( stderr goes to error.log)

ls missingfile 2>  /dev/null  (stderr goes to nothing)

 

cat file1 file2 nofile > mystdoutput 2>&1   (stderr goes to stdout file)

 

set -o     (shows options for shell)

set -o noclobber    (cant overwrite existing file)

set +o noclobber (turn it off)

 

PIPES (use output of one command as input to another command)

ls /etc/ |grep cron |grep daily > grep .txt

ls /etc |sort -f |less   (sort and display into less editor)

 

GREP, EGREP and FGREP

 

GREP

using grep for pattern matching - regular expressions

 

grep hello testfile.txt     …pulls out all hello words in our file

grep ^hello file  …all the hellos at the beginning of the line

grep hello$ file …lines end with hello

 

grep [hpok] file   - returns all lines with any of these chars in it

can use ^[] and []$ too

[a-g] or [1-9]

grep -f grepinput  file.txt    …use file grepinput as the pattern to look for in file.txt

grep -lr cron /etc    …list all files  in /etc/ recrusively that have cron in it.

 

EGREP - extended grep ..use it when we are using extra commands like |, .*

egrep is the same as writing  grep -e

egrep   'hello.*world' file.txt    (.* is and)

egrep -i      (ignore case)

'hello|world'   - search for lines with hello, or world or both

egrep -v    …returns everything that doesn't match

FGREP  - look for literal pattern, no extended patterns

fgrep is same as writing grep -F

 

fgrep hello$ file.txt    …find the lines with hello$  

 

CUT - take some of stdout

eg want to grab all usernames on the system

cut -f 1 -d: /etc/passwd   (fi = field 1,  d: = deliminator is :  )

 

 

               SED - stream editor

               eg

               sed 's/old/new/w newfile.txt'  oldfile.txt

 

               sed '0,/old/s/old/new' oldfile.txt    ..look for first occurance of old and only substitute that

 

               eg remove html tages

               sed 's/<[^]*>//' team      eg subsititute beginning with <, but not immeditaly followed by ^> , replace with blank

 

               TEE command - read from stdin and write to stdout

 

               eg want to show on screen and to file(s) at same time

ls | tee newfile1 newfile2

ls | tee -a newfile   …append

 

LINUX Text Editor : Vim (Vi iMproved)

VIM docs : http://www.vim.org/docs.php


sudo yum install vim   


cp $VIMRUNTIME/vimrc_example.vim ~/.vimrc        (sets up vim config file)


vimtutor         (20 minute hands on tutorial)

 

USEFUL commands:


               format:                operator number motion  

eg           d2w  = delete two words

 

               DELETE (CUT) / YANK (COPY)  - both these commands


               dw -delete to next word

               de - delete to end of this word

               d$ - delete to end of line

               dd  - delete whole line

               2dd  - delete two lines

               d2w - delete 2 words

               yy - YANK (cut the line)

               y2 -  cut two lines

               y5w - yank five words


PASTE


p -paste after cursor ,     (can past deleted words/lines)

P (capital P)- paste before cursor

              

UNDO/REDO


               u - undo last change

               U - undo all changes

               CTRL-R - redo

 

CHANGES


r - replace character

R - overwriting

ce - change till end of current word i.e. deletes word and lets you type in new one

               cw - change till beginning of next word

               c$ - change till end of line

               c2w -change 2 words

               I - insert before the cursor

               a - append after the cursor

               A -append at end of line

 

SEARCH


/  -forward search  , n - finds next , CTRL-o  goes back to previous found word

? -backwards search, n finds next

:set ic   (ignore case)

:set noic (no ignore case i.e turn ignore case off)

:set hls  (highlight search - all occurances highlighted)

:set nohls (turn highlight off)

:set ic (incremental search)

:set noic (turn off)

 

REPLACE


:s/old/new  - replace first one on current line

:s/old/new/g  - replace all on current line

:%s/old/new/g   - global replace

:%s/old/new/gc  - global replace and confirm first

 

MATCHING BRACKETS


put cursor on opening bracket, press % , goes to closing bracket

 

NAVIGATING


               2w - jump two words

               0 -beginning of line

CTRL-G displays file name,  current line number / total lines, %age f way thru doc

G move to end of file

5G - move to line 5

gg - move to first line

 

EXTERNAL COMMANDS:


!

!ls -l

 

SAVE/REMOVE FILE


!wq

!w test

v  (then highlight selection)   :w selection.txt

!rm test

 

INSERT AN EXTERNAL FILE


:r  text.txt

:r !ls


USE SHORTCUT TO ENTER COMMANDS


:e   CTRL-D  (shows all commands beginning with e) , press TAB - fills command

.   - repeats last command

3J - joins next three lines into one long line

 

3 Oct 2014

LINUX Text Editor : Vi

Vi (pronounced V I,  or vigh (rhymes with sigh) - very powerful text editor, standard one used across the industry. Small memory footprint , can be put onto recovery disks.

[Not Vim - is Vi iMproved  in newer distributions,  use that if its available).

3 modes to use Vi in:

               command mode : hit ESC  takes you to command mode can navigate the file

                              L H J K - move cursoe

                              yy or Y or 2yy - yank line 

                              p - paste underneath current line

                              P - paste on previous line

                              u - undo

                              G - move to  last line of file

5G - goto line 5

L - goto last line of terminal

H - goto top line of terminal

J - join lower line to current line

/   - searches left to right

?  -  searches right to left

               insert mode        :  entering text

                              i - insert

                              R - replace

                              cw - change word

                              cc - change line

               ex mode              :  hit ESC and press :   (colon) .

                              %s   - search and replace

                              w - write/save

                              q - quit,  wq!  - force quit

                              e   newfile.txt - edit a new file

                              r  oldfile.txt  - brings it in

                              !  run shell command eg ls /etc

RPM Based Package Management

DPKG command set and apt-get used by Debian/Ubuntu.


RPM and YUM used by  CentOS, Red hat, SUSE, Yellow Dog, Fedora


(Can convert using a tool called Alien).


 

RPM (Redhat Package Manager) is a package manager, but cannot resolve dependencies. 


YUM (YellowDog Update Modifier) is a frontend to RPM and can manage dependencies.

 

RPM Naming : package_name.versionnumber-buildnumber.architecture.rpm


Install YUM

               man yum  (for manual)

yum install wget

wget http://www.nano-editor.org/dist/v2.2/RPMS/nano-2.2.6-1.i386.rpm


Practise using rpm and yum

rpm -i nano-2.2.6-1.i386.rpm

But we already have a package called nano installed (whereis nano )

rpm -e nano    (extracts i.e. removes existing nano)

rpm -ihv nano-2.2.6-1.i386.rpm  (installs and shows progress but fails die to missing dependencies)

yum nano-2.2.6-1.i386.rpm    (works fine).

GET INFO

rpm -qi nano  - tells us the package name and details like install date

rpm -q --list nano    - all files installed with nano package

rpm -qR nano      -queries all Requirements i.e dependencies

rpm --rebuilddb   (cleans the rpm cache)

 

Global Configuration file for RPM is  in   cat /usr/lib/rpm/rpmrc   (don't need to change it)

tail /usr/lib/rpm/rpm.log    (shows wehre log file is)

vi /var/log/rpmpkgs  (can look at log file)

 

YUM Software Management

vi /etc/yum/yum.conf    - config file , can add new source repos here

 or in /etc/yum/repos.d   , each file is a separate repo that has sources.

INSTALL/UPDATE YUM

yum install

or yum update

CHECK FOR INSTALLED PACKAGE UPDATES

yum check-update

SEARCH

yum search http   (eg searching to install apache)

yum install http

yum check-update http

yum check-upgrade  (checks to see if yum needs upgrading)

yum info httpd.i686   (same as rmp qi)

yum deplist (dependency list)

can append --force , to force install even if dependencies are not there.

 

yumdownloader & rpm2cpio


yumdownloader--resolve  httpd (just download the rpm package and dependencies not install etc)


To view contents of the rpm use  the cpio  utility for std input and std output


rpm2cpio   package_name.rpm  |cpio -t       (converts rpm to std output (messy to read as its code), then the cpio -t lets us view it)