Administration – Reffffference https://ivan.reallusiondesign.com For my personal reference purposes Mon, 08 Jul 2019 15:24:42 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.5 106465655 Pipe mysqldump to ssh/sftp https://ivan.reallusiondesign.com/pipe-mysqldump-to-ssh-sftp/ https://ivan.reallusiondesign.com/pipe-mysqldump-to-ssh-sftp/#disqus_thread Mon, 08 Jul 2019 15:24:42 +0000 https://ivan.reallusiondesign.com/?p=961 If you need to send mysqldump over ssh on different server without creating any files on the local server, mostly if you are low on space you can use this:
mysqldump -u MYSQL_USERNAME -p DATABASE | gzip -c | ssh USER@HOST 'cat > ~/dump.sql.gz'


And for the restore from remote location:
ssh USER@HOST "cat /path/to/db.sql" | mysql -uUSER  -pPASSWORD DATABASE

]]>
https://ivan.reallusiondesign.com/pipe-mysqldump-to-ssh-sftp/feed/ 0 961
Enabling query cache for MariaDB/MySQL https://ivan.reallusiondesign.com/enabling-query-cache-for-mariadb-mysql/ https://ivan.reallusiondesign.com/enabling-query-cache-for-mariadb-mysql/#disqus_thread Thu, 27 Jun 2019 00:47:10 +0000 https://ivan.reallusiondesign.com/?p=955 Query cache can speed up to 2-3 times queries that are often run.
It is pretty easy to set up. First you need to check if query cache is supported for your system:

show variables like 'have_query_cache';

That should return:

+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| have_query_cache | YES   |
+------------------+-------+

Then you need to check some variables related to query caching. I will explain each of them:

mysql> show variables like 'query_cache_%' ;
+------------------------------+----------+
| Variable_name                | Value    |
+------------------------------+----------+
| query_cache_limit            | 262144   |
| query_cache_min_res_unit     | 4096     |
| query_cache_size             | 10485760 |
| query_cache_type             | ON       |
| query_cache_wlock_invalidate | OFF      |
+------------------------------+----------+

query_cache_limit – the total size each individual query can be. In my case that is 262144
query_cache_min_res_unit – how large is each chunk/block of cached data
query_cache_size – the total amount of cached queries, 0 disables it, and it needs to be at least 40kb at the lowest to work. Usually the default is 16,777,216
query_cache_type – if the value is ON it means cache query is enabled. If it is OFF it means it is disabled.

Add the following to your sql config file, probably in /etc/mysql/my.cnf and edit the values to fit your needs and/or your servers specs –

[mysqld]
query_cache_type=1
query_cache_size = 32M
query_cache_limit=512K

Then restart mysql/maridb and you should be good to go. You can verify the changes in the config file by running again:

show variables like 'query_cache_%';

You can check some query cache stats by running this:

mysql> SHOW STATUS LIKE 'Qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_free_blocks      | 2        |
| Qcache_free_memory      | 14713664 |
| Qcache_hits             | 1750     |
| Qcache_inserts          | 1643     |
| Qcache_lowmem_prunes    | 0        |
| Qcache_not_cached       | 247      |
| Qcache_queries_in_cache | 423      |
| Qcache_total_blocks     | 887      |
+-------------------------+----------+

For further and more detailed information, MariaDB knowledge base about Query Cache is far better place.

]]>
https://ivan.reallusiondesign.com/enabling-query-cache-for-mariadb-mysql/feed/ 0 955
CentOS Configure account to never expire / fix cron problem https://ivan.reallusiondesign.com/centos-configure-account-to-never-expire-fix-cron-problem/ https://ivan.reallusiondesign.com/centos-configure-account-to-never-expire-fix-cron-problem/#disqus_thread Fri, 15 Feb 2019 15:47:08 +0000 https://ivan.reallusiondesign.com/?p=941 CentOS have security measure to force users password expiration which can cause problems.
For example – you do not use passwords to log in to machines, and you prefer ssh keys. And the day the password for that user expires its cron jobs will stop working.

You can fix it with one of these, I prefer removing the expiration with chage:

chage -M -1 root

or

passwd -x -1 root

You can confirm with:

chage -l root
]]>
https://ivan.reallusiondesign.com/centos-configure-account-to-never-expire-fix-cron-problem/feed/ 0 941
MariaDB master-slave cluster on Ubuntu https://ivan.reallusiondesign.com/mariadb-master-slave-cluster-on-ubuntu/ https://ivan.reallusiondesign.com/mariadb-master-slave-cluster-on-ubuntu/#disqus_thread Fri, 08 Feb 2019 19:01:48 +0000 https://ivan.reallusiondesign.com/?p=924 This article explains how to run MariaDB SQL server in as master/slave replication cluster on two Ubuntu virtual machines.

master: 192.168.122.25
slave: 192.168.122.26

1. Before anything else you need to update all packages on the two machines:

sudo apt update
sudo apt upgrade

2. First thing is to add the official MariaDB repo for the stable release from here – https://downloads.mariadb.org/mariadb/repositories/
In my case, for Ubuntu 18.04 I had to use this:

sudo apt-get install software-properties-common
sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8
sudo add-apt-repository 'deb [arch=amd64,arm64,ppc64el] http://ams2.mirrors.digitalocean.com/mariadb/repo/10.3/ubuntu bionic main'

3. Install it on both servers:

sudo apt install mariadb-server

You will have to provide password for the root user during install. Please note this is not the exisitng Ubuntu root user, but is new password the root user for mysql.

4. On both servers: sudo mysql_secure_installation
This will ask you for the root password you have set up in the previous step. You should remove anonymous users, disable remote root loginand remove test database. Basically answer yes[Y] to all if you are installing this on a machine available from the internet.

5. On both servers:

sudo systemctl enable mariadb.service
sudo systemctl start mariadb

The first command will make the mariadb server start every time the machine is re/started and the second will just the start service right now as it still not running.

6. On the master server create empty database

MariaDB [(none)]> mysql -uroot -p
MariaDB [(none)]> create database database_name;

7. On the master server we need to enable binary logging.
– Backup the original file in /etc/mysql/

cp my.cnf my.cnf.bkp

Add this new lines under the [mysqld] section, and replace the IP address with the one your master machine have.

#Replication settings
log-bin
server_id=1
bind-address=192.168.122.25
binlog-ignore-db = information_schema
binlog-ignore-db = mysql
binlog-ignore-db = performance_schema
binlog-ignore-db = test

This will replicate all new databases to the slave server, if you like to replicate just one specific database you should use

replicate-do-db = 

8. Now we need to login to the master sql server and create replication user and give the necessary grants.

MariaDB [(none)]> CREATE USER 'slave'@'localhost' IDENTIFIED BY 'SomePassword';
MariaDB [(none)]> GRANT REPLICATION SLAVE ON *.* TO slave IDENTIFIED BY 'SomePassword' WITH GRANT OPTION;
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> FLUSH TABLES WITH READ LOCK;
MariaDB [(none)]> SHOW MASTER STATUS;

The last command output is important in order the slave to know from which point it should start replicating from.

Unlock the databases and exit:

MariaDB [(none)]> UNLOCK TABLES;
MariaDB [(none)]> exit;

9. Login to the slave and create another empty database with the same name and the slave user.

CREATE DATABASE DATABASE_NAME;
CREATE USER 'slave'@'localhost' IDENTIFIED BY 'SomePassword';
FLUSH PRIVILEGES;

10. Add this the to the [mysqld] section in /etc/mysql/my.cnf in the slave:

server_id=2

Note that the master have server_id=1 so you should have different IDs on the different servers.

11. Log in to the slave database and run the following commands in the MariaDB prompt. Double check the MASTER_LOG_FILE and MASTER_LOG_POS variables, which should be the same as the values returned by SHOW MASTER STATUS above.

MariaDB [(none)]> CHANGE MASTER TO
MASTER_HOST='192.168.122.25',
MASTER_USER='slave',
MASTER_PASSWORD='SomePassword',
MASTER_PORT=3306,
MASTER_LOG_FILE='mariadb-bin.000001',
MASTER_LOG_POS=314,
MASTER_CONNECT_RETRY=10,
MASTER_USE_GTID=current_pos;

Now start the slave and check the status without exiting the MariaDB prompt:

MariaDB [(none)]> START SLAVE;
MariaDB [(none)]> SHOW SLAVE STATUS\G;

12. Test the replication:
login in the master server and create table in our empty database:

CREATE TABLE IF NOT EXISTS names (
task_id INT AUTO_INCREMENT,
title VARCHAR(255) NOT NULL,
start_date DATE,
due_date DATE,
status TINYINT NOT NULL,
priority TINYINT NOT NULL,
description TEXT,
PRIMARY KEY (task_id)
) ENGINE=INNODB;

You should see the new table created on the slave server too.

13. Debug: If there is something wrong with the slave replication it should show with when you run

SHOW SLAVE STATUS\G;

Most of the time problems are easily resolved with updating the slave configuration with the

CHANGE MASTER

query, stopping and then starting the slave. Watch for log position and the log file name.

]]>
https://ivan.reallusiondesign.com/mariadb-master-slave-cluster-on-ubuntu/feed/ 0 924
Top linux commands to use. https://ivan.reallusiondesign.com/top-linux-commands-to-use/ https://ivan.reallusiondesign.com/top-linux-commands-to-use/#disqus_thread Mon, 24 Dec 2018 08:27:44 +0000 https://ivan.reallusiondesign.com/?p=900 Table with the command I most often use and think are essential. Or some that are cool, but easy to forget/not so often used.

ctrl+rSearch in bash history
ctrl+eGo to the end of the line
ctrl+uCut the characters before the cursor
ctrl+yYank/paste, it can paste what you cut with ctrl+u
ctrl+shift+cCopy the marked text
ctrl+shift+vPaste the text from the previous command
ctrl+dClose bash sessions, same as to type exit
!$Get the last argument from the previous command
!*Get all the arguments from the previous command
historyDon’t add command to bash history. There is space in front of the command
disown -a && exitExit terminal, detach all background process, so they can run. Useful for long tasks.
fcOpen last command in editor. Fix very long one-liners if you mess them up.
ctrl+x+eCompose command in the default editor and execute it on save.
curl ifconfig.me Get your public IP from CLI.
very_long_command # labelLabel long commands, so it it easier to find in history. Everything after # is not executed as it is bash comment.
rm !(*.foo|*.bar|*.html) Remove all files except the ones with these extensions.
vim -x <FILENAME>Encrypt file in vim.
man hierShow filesystem hierarchy.
cat /etc/issue Get distro name.
ps aux | grep [p]rocess-name Find the process you are looking for, without showing the grep command itself.
]]>
https://ivan.reallusiondesign.com/top-linux-commands-to-use/feed/ 0 900
Mount directory into the RAM https://ivan.reallusiondesign.com/mount-directory-into-the-ram/ https://ivan.reallusiondesign.com/mount-directory-into-the-ram/#disqus_thread Thu, 05 Jul 2018 20:20:44 +0000 https://ivan.reallusiondesign.com/?p=875 Needed to mount WordPress cache folder into the RAM of one VPS, to get that little bit of extra speed.
To mount it temporary and see how it works for you, you can use:

mount -t tmpfs -o size=64M tmpfs /absolute/path/to/your/folder/

To make it permanent you need to add this in the /etc/fstab file:

tmpfs /absolute/path/to/your/folder tmpfs defaults,size=64M 0 0
]]>
https://ivan.reallusiondesign.com/mount-directory-into-the-ram/feed/ 0 875
Systemd simple service https://ivan.reallusiondesign.com/systemd-simple-service/ https://ivan.reallusiondesign.com/systemd-simple-service/#disqus_thread Sat, 09 Jun 2018 22:46:13 +0000 https://ivan.reallusiondesign.com/?p=872 This a template for simple sysmtemd service to change the ownership of a file, since in my case the file is in /sys and it is generated on boot, so using acl didn’t help me. I had to use this hack to change the ownership of a file on each boot.
What I need is a write permissions to a file in order to change the brightness on my laptop with i3wm.

The file is /etc/systemd/system/brightness.service
but there is symlink from the /etc/systemd/system/multi-user.target.wants directory.

This are the contents of the service file:

[Unit]
Description=Alter permissionsfor brightness

[Service]
ExecStart=/bin/chmod go+rw /sys/class/backlight/intel_backlight/brightness
ExecStop=/bin/chmod go+rw /sys/class/backlight/intel_backlight/brightness

[Install]
WantedBy=multi-user.target

You will also want to enable and start the service with:

systemctl enable brightness.service
systemctl start brightness.service

All of this action is happening on Ubuntu 18.04

]]>
https://ivan.reallusiondesign.com/systemd-simple-service/feed/ 0 872
Remove files that are not in a list https://ivan.reallusiondesign.com/remove-files-that-are-not-in-a-list/ https://ivan.reallusiondesign.com/remove-files-that-are-not-in-a-list/#disqus_thread Thu, 31 May 2018 20:10:27 +0000 https://ivan.reallusiondesign.com/?p=861 Needed to remove bunch of files that are not matching list given to me, so to leave only the files that are on the list, which turned out to be a little hard, or at least for me.

That did the trick:

 find . -name "*" | grep -vFf update_images.csv | xargs rm -f

If the file names contain white spaces, you will have to use the following command, but I haven’t tested it myself:

find . -name "*" | grep -vFf update_images.csv |sed 's/^/"/;s/$/"/' | xargs rm -rf
]]>
https://ivan.reallusiondesign.com/remove-files-that-are-not-in-a-list/feed/ 0 861
MySQL tuning for low memory servers https://ivan.reallusiondesign.com/mysql-tuning-for-low-memory-servers/ https://ivan.reallusiondesign.com/mysql-tuning-for-low-memory-servers/#disqus_thread Sun, 18 Feb 2018 04:17:03 +0000 https://ivan.reallusiondesign.com/?p=847 If you are running couple of sites on a small VPS and you have 512-1024MB RAM, you are probably not amazed by mysql eating 400 just to start.
You can cut 200-300 of that memory usage by disabling performance schema in your config file – usually called my.cnf

performance_schema = off

If you are wondering if you should disable it and what will happen – you most probably don’t need that feature so it is safe to disable it. In a few words – it is to help you tune the SQL server, queries, find bottlenecks etc.
You can get more info here – MySQL performance schema

]]>
https://ivan.reallusiondesign.com/mysql-tuning-for-low-memory-servers/feed/ 0 847
php-fpm child processes memory usage https://ivan.reallusiondesign.com/php-fpm-child-processes-memory-usage/ https://ivan.reallusiondesign.com/php-fpm-child-processes-memory-usage/#disqus_thread Thu, 11 Jan 2018 17:44:10 +0000 https://ivan.reallusiondesign.com/?p=835 Often you need to adjust php-fpm for the memory available, and to do so you need to average the child processes memory usage.

Human readable:

 ps -eo size,pid,user,command --sort -size | awk '{ hr=$1/1024 ; printf("%13.2f Mb ",hr) } { for ( x=4 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }' | grep php-fpm 

and also machine friendly:

 ps -ylC php-fpm --sort:rss 
]]>
https://ivan.reallusiondesign.com/php-fpm-child-processes-memory-usage/feed/ 0 835