Category Archives: Linux

Remove files that are not in a list

Needed to remove bunch of files that are not matching list given to me, so to leave only the files that are on the list, which turned out to be a little hard, or at least for me.

That did the trick:

 find . -name "*" | grep -vFf update_images.csv | xargs rm -f

If the file names contain white spaces, you will have to use the following command, but I haven’t tested it myself:

find . -name "*" | grep -vFf update_images.csv |sed 's/^/"/;s/$/"/' | xargs rm -rf

MySQL tuning for low memory servers

If you are running couple of sites on a small VPS and you have 512-1024MB RAM, you are probably not amazed by mysql eating 400 just to start.
You can cut 200-300 of that memory usage by disabling performance schema in your config file – usually called my.cnf

performance_schema = off

If you are wondering if you should disable it and what will happen – you most probably don’t need that feature so it is safe to disable it. In a few words – it is to help you tune the SQL server, queries, find bottlenecks etc.
You can get more info here – MySQL performance schema

php-fpm child processes memory usage

Often you need to adjust php-fpm for the memory available, and to do so you need to average the child processes memory usage.

Human readable:

 ps -eo size,pid,user,command --sort -size | awk '{ hr=$1/1024 ; printf("%13.2f Mb ",hr) } { for ( x=4 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }' | grep php-fpm 

and also machine friendly:

 ps -ylC php-fpm --sort:rss 

Fix i3wm tearing in ubuntu

i3 is great window manger, but for some time I had struggles fixing some tearing it had, until I found recently this answer, which fixed it for me.

You will have to install the comptom composite manager with:

sudo apt install compton

then use the following config in ~./config/compton.conf or wherever you prefer to keep you config files. Then place this in that config file:

# basic configuration
backend = "glx";
vsync = "opengl-swc";

glx-copy-from-front = true;
glx-swap-method = 2;
xrender-sync = true;
xrender-sync-fence = true;

# transparancy settings for i3
opacity-rule = [
    "0:_NET_WM_STATE@:32a *= '_NET_WM_STATE_HIDDEN'"
];

Or this variation:

backend = "glx";
glx-no-stencil = true;
paint-on-overlay = true;
vsync = "opengl-swc";

Now you can start compton with that config file to test if it solves the problem for you:

compton --config ~/.config/compton.conf -b 

If that work for you, you can place it in your i3wm config file, so it will be loaded on startup –

  exec --no-startup-id compton --config ~/.config/compton.conf -b

Sed replace in file

This one liner will replace all occurrences in given file.

sed -i -e 's#https://www.oldurl.xyz/#http://newurl.us#g' dump.sql 

Sed can use for delimiter anything you specify, so in the above example is #(hashtag jajajaj :D)
But it could be comma or whatever fits your case, so this will work too:

sed -i -e 's,https://www.oldurl.xyz/,http://newurl.us,g' dump.sql 

osTicket nginx config

Had to install osTicket recently, and it had bit of a problems with the ajax requests returning 404.
This config should be enough to get you started, you might not even need to change it, well besides the obvious things like server name and root.
Ideally it shouldn’t have if blocks, but I am too lazy right now to refine it.

server {
        root /var/www/osticket/;
        index index.php index.html;
        listen 80;

        server_name domain.com;

        set $path_info "";

        # Deny access to everything inside the include directory
        location ~ ^/include {
                deny all;
                return 403;
        }

        # Deny access to .htaccess
        location ~ /\.ht {
                deny all;
        }

        # Requests to /api/* need their PATH_INFO set, this does that
        if ($request_uri ~ "^/api(/[^\?]+)") {
                set $path_info $1;
        }

        # /api/*.* should be handled by /api/http.php if the requested file does not exist
        location ~ ^/api/(tickets|tasks)(.*)$ {
                try_files $uri $uri/ /api/http.php;
        }

        # /scp/ajax.php needs PATH_INFO too
        if ($request_uri ~ "^/scp/.*\.php(/[^\?]+)") {
                set $path_info $1;
        }

        # Catch requests to /scp/ajax.php/some/path and redirect them to ajax.php
        location ~ ^/scp/ajax.php/(.*)$ {
                try_files $uri $uri/ /scp/ajax.php;
        }

        # Set index.php as directory index
        location / {
                index index.php;
        }

        # PHP-FPM listening on 127.0.0.1:9001 or on a socket
        location ~ \.php$ {
                try_files $uri =404;
                fastcgi_pass 127.0.0.1:9001;
                #fastcgi_pass    unix:/var/run/php5-fpm.sock;
                fastcgi_index   index.php;
                fastcgi_param   SCRIPT_FILENAME         $document_root$fastcgi_script_name;
                fastcgi_param   PATH_INFO               $path_info;
                include fastcgi_params;
        }
}

Mount SSHFS volumes in fstab with ssh key

SSHFS on command line usually takes the ssh key with the -o, option which doesn’t really work when adding it in fstab. If you look around the internet you most probably found that people recommend adding fstab records with the -o option, but this wont work, simply add the key as another regular fstab option –

sshfs#USER@domain.com:/data/www /mnt/logs/  fuse IdentityFile=/home/USER/.ssh/id_rsa,uid=UID,gid=GUID,users,idmap=user,noatime,allow_other,_netdev,reconnect,ro 0 0 

Replace USER with the user who connects to the remote server and UID and GUID with the ones from the remote server.
The above also mounts the remote system as read-only so you wont be able to write on the mount.

No space left on device.

Sometimes we can be fooled by error messages. For example one sunny day you see that for some reason your web or mail server doesn’t work. So you go to check the logs and find something similar to this:

2016/12/28 09:02:37 [crit] 24668#24668: *472674 open() "/var/cache/nginx/client_temp/0020878597" failed (28: No space left on device), client: 192.168.1.1, server: www.domain.com, request: "GET /cart/add/uenc/aHR0cDovL3d3dy5hYmNob21lLmNvbS9zaG9wL2xvdi1vcmdhbmljLWxvdi1pcy1iZWF1dGlmdWwtdGVh/product/19471/form_key/N8l3OyVkC1el9T8q/?product=19471&related_product=&send_to_friend=%2F%2Fwww.domain.com%2Fshop%2Fsendfriend%2Fproduct%2Fsend%2Fid%2F19471%2F&form_key=N8l3OyVkC1el9T8q&super_group%5B19425%5D=1&super_group%5B19424%5D= HTTP/1.1", host: "www.domain.com", referrer: "http://www.domain.com/shop/organic-tea"

Then when you check the free space you see that you have more than enough, and all kind of irrational thoughts start flowing into your mind, when it is the simple inodes space.

Usually it is just that there is not enough inodes left free on your files system, simple as that, but is easy to overlook as for some people this doesn’t happen often (and it shouldn’t).

[root@hostname client_temp]# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/os-root 1703936 1703103 833 100% /
tmpfs 1524264 4 1524260 1% /dev/shm
/dev/sda1 51000 50 50950 1% /boot
/dev/mapper/os-tmp 131072 2155 128917 2% /tmp
/dev/mapper/data-data
19660800 578302 19082498 3% /data

Nginx basic authentication

Setting up basic authentication in Nginx is pretty easy, you need to first add couple of directives in block config, you can out them directly in the server if you want the whole site not be accessible or just on some parts-

    auth_basic "Restricted Content";
    auth_basic_user_file /etc/nginx/.htpasswd;

Then in the .htpasswd file add the user and use openssl to generate hash for the chosen password –

echo "password" | openssl passwd -apr1 -stdin