Terrible English by PT sir of a school in village:

Terrible English by PT sir of a school in village: 1) There is no wind in the football..
2) I talk, he talk, why you middle talk? .
3) You rotate the ground 4 times..
4) You go and understand the tree.
5) I’ll give you clap on ur cheeks..
6) Bring your parents and your mother and especially your father.
7) Close the window airforce is coming.
8) I have two daughters and both are girls..
9) Stand in a straight circle..
10) Don’t stand in front of my back
11) Why Haircut not cut..?
12) Don’t make noise.. principle is rotating in the corridor
13) Why are you looking at the monkey outside the window when I’m here?
14) You talking bad habit
15) Give me a red pen of any colour.
16) Can i have some snow in my cold drink?
17) Pick the paper and fall into the dustbin.
18) Both of u stand together separately.
19) Keep quiet the principal just passed away!!

Dont laugh alone pass it….

Using GPG

As an example let’s say we want to verify integrity of following file from https://hashcat.net/hashcat/

1. First download the file and its PGP signature (ends with .asc)

$ wget https://hashcat.net/files/hashcat-6.1.1.7z.asc --no-check-certificate
$ wget https://hashcat.net/files/hashcat-6.1.1.7z --no-check-certificate

2. Next download and import the key from PGP keyserver with following command:

$ gpg --keyserver keyserver.ubuntu.com  --recv-keys 8A16544F
gpg: requesting key 8A16544F from hkp server keyserver.ubuntu.com
gpg: /home/sjain68/.gnupg/trustdb.gpg: trustdb created
gpg: key 8A16544F: public key "Hashcat signing key <signing@hashcat.net>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

3. Now test the signature

$ gpg --verify hashcat-6.1.1.7z.asc hashcat-6.1.1.7z
gpg: Signature made Wed 29 Jul 2020 06:25:34 AM EDT using RSA key ID 8A16544F
gpg: Good signature from "Hashcat signing key <signing@hashcat.net>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: A708 3322 9D04 0B41 99CC  0052 3C17 DA8B 8A16 544F

To make the warning disappear we need to sign hashcat’s key with our private key. That will make it trusted. See https://security.stackexchange.com/a/147467/44139

Migrating Docker Volumes

By default Docker stores its persistent data in /var/lib/docker folder on Linux systems. This folder is part of the OS disk and in a production (or even dev) setup you might want to store persistent data on a data disk. Many OS Disks come with limited size. On a server that I was working on the /var folder only had 8GB to it.

/dev/mapper/rootvg-varlv   7.8G  7.8G     0 100% /var

It got full fast and then we had a big problem. We had running containers on the machine. What to do now? First we had to attach a data disk to the machine. Once that is done follow the recipe below to migrate Docker’s storage to another folder while keeping everything intact! The instructions below are for Red Hat Enterprise Linux (RHEL) and so should work as-is on CentOS. On other systems you might need some tweaks to the commands.

The new location is /app which resides on a different disk in this writeup.

Step 1: Stop Docker daemon

#-> systemctl stop docker

Step 2: Copy /var/lib/docker to new location

Copy the whole of /var/lib/docker not just the volumes inside it or you will lose your containers etc.

#-> rsync -aP /var/lib/docker /app

Run ls /app/docker to verify everything is there.

#-> ls /app/docker
builder   containerd  image    overlay2  runtimes  tmp    volumes
buildkit  containers  network  plugins   swarm     trust

Step 3: Rename /var/lib/docker to /var/lib/docker.old

For safety, always retain the old directory.

#-> mv /var/lib/docker /var/lib/docker.old

To see the file size

#-> du -ch -d 1 /var/lib/docker.old
212K	/var/lib/docker.old/containerd
1.1G	/var/lib/docker.old/overlay2
72K	/var/lib/docker.old/buildkit
1.5M	/var/lib/docker.old/containers
20K	/var/lib/docker.old/builder
62M	/var/lib/docker.old/swarm
4.0K	/var/lib/docker.old/tmp
24K	/var/lib/docker.old/plugins
995M	/var/lib/docker.old/volumes
4.0K	/var/lib/docker.old/runtimes
4.0K	/var/lib/docker.old/trust
372K	/var/lib/docker.old/network
9.5M	/var/lib/docker.old/image
2.1G	/var/lib/docker.old
2.1G	total

WARNING: If you run mv /var/lib/docker /app/docker.old its going to hang for a while because /app is on another filesystem and so it has to really move the files to that folder – it can’t just rename the folder. So don’t do that. Run rsync again if you want to keep an archive of the data on /app.

Step 4: Add symlink from /var/lib/docker to /app/docker

Note the destination folder comes as the first argument to the ln command:

#-> ln -s /app/docker /var/lib/docker

The old containers have /var/lib/docker paths configured in them and so we need to setup this symlink so that we can restart the old containers and everything works.

Verify:

#-> ls -al /var/lib/docker
lrwxrwxrwx 1 root root 11 Sep 29 17:36 /var/lib/docker -> /app/docker

This is the error I got when trying to restart the container if I skipped this step and did not setup the symlink:

#-> docker start middleware-mysql
Error response from daemon: error evaluating symlinks from mount source "/var/lib/docker/volumes/middleware-mysql/_data": lstat /var/lib/docker/volumes: no such file or directory
Error: failed to start containers: middleware-mysql

Step 5: Edit /lib/systemd/system/docker.service and set the --data-root variable

The --data-root is explained below:

--data-root string                      Root directory of persistent Docker state (default "/var/lib/docker")

Edit following file:

#-> vi /lib/systemd/system/docker.service

and add entry for --data-root pointing it to /app/docker:

ExecStart=/usr/bin/dockerd --data-root /app/docker

Step 6: Reload configuration

#-> systemctl daemon-reload

Step 7: Restart Docker

#-> systemctl start docker

Run systemctl status docker to verify everything is good

Step 8: Restart containers

Finally just restart the containers:

#-> docker start foo
#-> docker start bar

That’s it! Everything is restored.

It was a nerve-racking experience for sure. Now I saw following when I did a df -h

#-> df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   3.9G     0  3.9G   0% /dev
tmpfs                      3.9G  4.0K  3.9G   1% /dev/shm
tmpfs                      3.9G  378M  3.6G  10% /run
tmpfs                      3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/rootvg-rootlv  7.8G  145M  7.2G   2% /
/dev/mapper/rootvg-usrlv   9.8G  2.1G  7.2G  23% /usr
/dev/sda1                  976M   83M  826M  10% /boot
/dev/sdc                   246G  2.1G  232G   1% /app
/dev/mapper/rootvg-homelv  976M  364M  546M  40% /home
/dev/mapper/rootvg-optlv   2.0G  1.6G  280M  85% /opt
/dev/mapper/rootvg-varlv   7.8G  7.5G     0 100% /var
/dev/mapper/rootvg-tmplv   2.0G  379M  1.5G  21% /tmp
/dev/sdb1                   16G  2.1G   13G  14% /mnt/resource
tmpfs                      797M     0  797M   0% /run/user/48081
overlay                    246G  2.1G  232G   1% /app/docker/overlay2/b65f1433823722f8bba50ff4461f8a8789a9f652f33532326c96bc6215af9d28/merged
shm                         64M     0   64M   0% /app/docker/containers/cf51fefa07b118eb252b382947650ad3fbd58a2dbeb2cd7c72e23ab1a0f9e862/mounts/shm
overlay                    246G  2.1G  232G   1% /app/docker/overlay2/ca0999cd7dee9d9b3a9911150ff4e6bc06d8c5552a9faa2158f3ecc35cce3a9f/merged
shm                         64M     0   64M   0% /app/docker/containers/1f1ff1dc1e4cf96decdd6ea57fe51a72c1edbfdad89771e396f217fc5d294383/mounts/shm

Understanding Internals of Chaincode getState/putState calls

If we look at the ChaincodeStub class in $/hyperledger/fabric-chaincode-node/libraries/fabric-shim/lib/stub.js we can see following code

async getState(key) {
        logger.debug('getState called with key:%s', key);
        // Access public data by setting the collection to empty string
        const collection = '';
        return await this.handler.handleGetState(collection, key, this.channel_id, this.txId);
    }

    async putState(key, value) {
        // Access public data by setting the collection to empty string
        const collection = '';
        if (typeof value === 'string') {
            value = Buffer.from(value);
        }
        return await this.handler.handlePutState(collection, key, value, this.channel_id, this.txId);
    }

The code for the handler (which is of type ChaincodeSupportClient) can be found in $/hyperledger/fabric-chaincode-node/libraries/fabric-shim/lib/handler.js

async handleGetState(collection, key, channel_id, txId) {
        const msg = {
            type: fabprotos.protos.ChaincodeMessage.Type.GET_STATE,
            payload: fabprotos.protos.GetState.encode({key, collection}).finish(),
            txid: txId,
            channel_id: channel_id
        };
        logger.debug('handleGetState - with key:', key);
        return await this._askPeerAndListen(msg, 'GetState');
    }

    async handlePutState(collection, key, value, channel_id, txId) {
        const msg = {
            type: fabprotos.protos.ChaincodeMessage.Type.PUT_STATE,
            payload: fabprotos.protos.PutState.encode({key, value, collection}).finish(),
            txid: txId,
            channel_id: channel_id
        };
        return await this._askPeerAndListen(msg, 'PutState');
    }

and it looks like this handler is instantiated in $/hyperledger/fabric-chaincode-node/libraries/fabric-shim/lib/chaincode.js

static start(chaincode) {
...
const client = new Handler(chaincode, url, optsCpy);

which in turn gets called when the chaincode is registered in $/hyperledger/fabric-chaincode-node/libraries/fabric-shim/lib/contract-spi/bootstrap.js

static register(contracts, serializers, fileMetadata, title, version) {
        // load up the meta data that the user may have specified
        // this will need to passed in and rationalized with the
        // code as implemented
        const chaincode = new ChaincodeFromContract(contracts, serializers, fileMetadata, title, version);

        // say hello to the peer
        shim.start(chaincode);
    }

PHP Interactive Shell

I keep forgetting this so many times that I need to make a note of this. This is how you use it:

$ php -a
Interactive shell

php > $a='hello world';
php > echo $a;
hello world

Don’t forget the semicolons and remember PHP variables start with a $

Useful SO Queries

CDF of User Reputation

https://data.stackexchange.com/stackoverflow/query/1265631/cdf-of-so-reputation#graph

with cte as (select reputation, cume_dist() over (order by reputation) as cdf
from users)
select reputation, max(cdf) from cte
group by reputation
order by reputation;

According to this,

• 71% of users have reputation <= 1. These are users who just created an account on the site.
• 74% of users have reputation <= 10
• 92% of users have reputation <= 100
• 98% of users have reputation <= 1,000
• 99.8% of users have reputation <= 10,000

also: http://rjbaxley.com/posts/2016/11/08/Stack_Exchange_Reputation_Power_Law.html

Question Count over time

https://data.stackexchange.com/stackoverflow/query/1290036/number-of-questions-asked-over-time-for-a-particular-topic-tag?Tag1=hyperledger-fabric#graph

DECLARE @Tag1 varchar(255)  = ##Tag1:string##;

    SELECT      DATEADD (month, DATEDIFF (month, 0, q.CreationDate), 0)  AS [Month],
    
    COUNT (q.Id)                AS NumQuests

    FROM        Posts           q
    INNER JOIN  PostTags        pt
    ON          q.Id            = pt.PostId
    INNER JOIN  Tags            t
    ON          t.Id            = pt.TagId

    WHERE       q.PostTypeId    = 1
    AND         q.CreationDate >= '2016-01-01 00:00:00'
    AND         t.TagName       IN (@Tag1)

    GROUP BY    DATEDIFF (month, 0, q.CreationDate), 
                t.TagName
    order by    DATEDIFF (month, 0, q.CreationDate)

below graph shows number of questions tagged hyperledger-fabric over time

number of questions tagged hyperledger-fabric over time.

CDF of Question Count vs. time

https://data.stackexchange.com/stackoverflow/query/1165722/question-count-or-score-growth-over-time-by-tag-comparison?ShowScore=0&Tag1=hyperledger-fabric&Tag2=quorum&Tag3=bigchaindb&Tag4=corda&Tag5=tendermint&Tag6=hyperledger-sawtooth#graph

total number of questions over time

failed to create runc console socket: mkdir /tmp/pty064803763: no space left on device: unknown

Got this error today when trying to log into a docker container

$ docker exec -it mysql /bin/bash
failed to create runc console socket: mkdir /tmp/pty064803763: no space left on device: unknown

Then I tried

$ df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   3.9G     0  3.9G   0% /dev
tmpfs                      3.9G  4.0K  3.9G   1% /dev/shm
tmpfs                      3.9G  378M  3.6G  10% /run
tmpfs                      3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/rootvg-rootlv  7.8G  128M  7.3G   2% /
/dev/mapper/rootvg-usrlv   9.8G  2.3G  7.0G  25% /usr
/dev/sda1                  976M  113M  797M  13% /boot
/dev/mapper/rootvg-optlv   2.0G  1.5G  303M  84% /opt
/dev/mapper/rootvg-homelv  976M  3.9M  905M   1% /home
/dev/mapper/rootvg-tmplv   2.0G  432M  1.4G  24% /tmp
/dev/mapper/rootvg-varlv   7.8G  4.2G  3.3G  56% /var
/dev/sdc                   246G  1.1G  233G   1% /app
/dev/sdb1                   16G  2.1G   13G  14% /mnt/resource
tmpfs                      797M     0  797M   0% /run/user/48081

which did not indicate any problem. /tmp is only 24% used.

$ df -i
Filesystem                  Inodes  IUsed    IFree IUse% Mounted on
devtmpfs                   1016516    431  1016085    1% /dev
tmpfs                      1019390      2  1019388    1% /dev/shm
tmpfs                      1019390    856  1018534    1% /run
tmpfs                      1019390     16  1019374    1% /sys/fs/cgroup
/dev/mapper/rootvg-rootlv   524288   6703   517585    2% /
/dev/mapper/rootvg-usrlv    655360  86575   568785   14% /usr
/dev/sda1                    65536    343    65193    1% /boot
/dev/mapper/rootvg-optlv    131072  53934    77138   42% /opt
/dev/mapper/rootvg-homelv    65536    166    65370    1% /home
/dev/mapper/rootvg-tmplv    131072 131072        0  100% /tmp
/dev/mapper/rootvg-varlv    524288  26116   498172    5% /var
/dev/sdc                  16384000   3649 16380351    1% /app
/dev/sdb1                  1048576     13  1048563    1% /mnt/resource
tmpfs                      1019390      1  1019389    1% /run/user/48081

Above is showing all the inodes have been exhausted under /tmp. On listing /tmp I saw lot of files like this

#-> cat /tmp/tmp.01amFL2hH4
read_kt /etc/krb5.keytab
write_kt /tmp/tmp.kt

For now I manually deleted the files. And after that it works

$ docker exec -it mysql /bin/bash
root@48b2dea3706b:/home#

Understanding the internals of wordpress php-fpm-alpine image

This post describes the internals of wordpress:php7.4-fpm-alpine Docker image.

The wordpress container does nothing but run the following command and wait for it to end which it never does as it spins up a server that listens for incoming connections forever:

$ docker-entrypoint.sh php-fpm

The file docker-entrypoint.sh can be found in /usr/local/bin/docker-entrypoint.sh inside the container. It is instructive to read and understand this file. It will do some setup etc. such as installing wordpress files in the WORKDIR which defaults to /var/www/html and ends with

exec "$@"

what above line does is to run php-fpm as if it were a continuation of the docker-entrypoint.sh script [1].

php-fpm is PHP’s Fast CGI processor which spins up a server listening at port 9000.

By inspecting the docker image (docker image inspect wordpress:php7.4-fpm-alpine) one can see from where the script installs PHP. This is given by the PHP_URL in below:

"Env": [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "PHPIZE_DEPS=autoconf \t\tdpkg-dev dpkg \t\tfile \t\tg++ \t\tgcc \t\tlibc-dev \t\tmake \t\tpkgconf \t\tre2c",
    "PHP_INI_DIR=/usr/local/etc/php",
    "PHP_EXTRA_CONFIGURE_ARGS=--enable-fpm --with-fpm-user=www-data --with-fpm-group=www-data --disable-cgi",
    "PHP_CFLAGS=-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64",
    "PHP_CPPFLAGS=-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64",
    "PHP_LDFLAGS=-Wl,-O1 -pie",
    "GPG_KEYS=42670A7FE4D0441C8E4632349E4FDC074A4EF02D 5A52880781F755608BF815FC910DEB46F53EA312",
    "PHP_VERSION=7.4.5",
    "PHP_URL=https://www.php.net/get/php-7.4.5.tar.xz/from/this/mirror",
    "PHP_ASC_URL=https://www.php.net/get/php-7.4.5.tar.xz.asc/from/this/mirror",
    "PHP_SHA256=d059fd7f55bdc4d2eada15a00a2976697010d3631ef6f83149cc5289e1f23c2c",
    "PHP_MD5=",
    "WORDPRESS_VERSION=5.4",
    "WORDPRESS_SHA1=d5f1e6d7cadd72c11d086a2e1ede0a72f23d993e"
]

Also we can see from PHP_EXTRA_CONFIGURE_ARGS that the php-fpm process will run as www-data. There are also references to www-data in docker-entrypoint.sh.

TIP: If you want to change the directory in which wordpress gets installed, simply change the –workdir to the location in which you want wordpress to be installed.

WordPress Boot Sequence

There are two great articles explaining how wordpress loads up when a request is received:

• https://theme.fm/wordpress-internals-how-wordpress-boots-up-part-2/
• https://theme.fm/wordpress-internals-how-wordpress-boots-up-part-3/

The bad: it looks like this boot sequence happens on EVERY request. That is why you don’t need to refresh or restart anything if you make changes to wp-config.php for example.

The boot sequence is this. Under /var/www/html there is an index.php which loads wp-blog-header.php which loads wp-load.php which loads wp-config.php which loads wp-settings.php which loads as many as 36 files.

bash-5.0# grep -n wp-blog-header.php index.php
4: * wp-blog-header.php which does and tells WordPress to load the theme.
17:require __DIR__ . '/wp-blog-header.php';
bash-5.0# grep -n wp-load.php wp-blog-header.php
13:	require_once __DIR__ . '/wp-load.php';
bash-5.0# grep -n wp-config.php wp-load.php
4: * and loading the wp-config.php file. The wp-config.php
8: * If the wp-config.php file is not found then an error
10: * wp-config.php file.
12: * Will also search for wp-config.php in WordPress' parent
27: * If wp-config.php exists in the WordPress root, or if it exists in the root and wp-settings.php
28: * doesn't, load wp-config.php. The secondary check for wp-settings.php has the added benefit
34:if ( file_exists( ABSPATH . 'wp-config.php' ) ) {
37:	require_once ABSPATH . 'wp-config.php';
39:} elseif ( @file_exists( dirname( ABSPATH ) . '/wp-config.php' ) && ! @file_exists( dirname( ABSPATH ) . '/wp-settings.php' ) ) {
42:	require_once dirname( ABSPATH ) . '/wp-config.php';
76:		/* translators: %s: wp-config.php */
78:		'<code>wp-config.php</code>'
83:		__( 'https://wordpress.org/support/article/editing-wp-config-php/' )
86:		/* translators: %s: wp-config.php */
88:		'<code>wp-config.php</code>'
bash-5.0# grep -n wp-settings.php wp-config.php
108:require_once ABSPATH . 'wp-settings.php';

Request Flow

in my nginx conf, I have

location / {
    try_files $uri $uri/ /index.php?$args;
}

Now when a request comes in for https://mywordpresssite.com/2020/04/22/e2e-connected-visibility-platform/ nginx will first see if there is a resource with that name. There isn’t any so nginx will perform internal redirect and attempt to serve index.php?2020/04/22/e2e-connected-visibility-platform. This will kick-in the index.php under /var/www/htmland the wordpress boot sequence. So requests for pretty much all resources are going through index.php which acts as the Main method.

Debugging File Upload Issues with WordPress

First make sure you have adequately increased the maximum file size that can be uploaded to wordpress. It is possible that uploads may still not work. On php-fpm-alpine image (and even on apache) wordpress runs under the credentials of www-data user. This user needs to have write access to the wp_upload_dir directory and the subdirectory under it where it will attempt to create the file. The full path resolves to /var/www/html/wp-content/uploads/2020/06 to give an example. Use the namei utility with the -om flag to see if www-data has sufficient permissions to be able to create a file in the directory. Use chown -R to make www-data the owner of all subdirectories under wp-content.

How to change the maximum file size you can upload to wordpress on nginx w/ php-fpm

Apache is a solid web server but my goto environment for wordpress has now changed to running it on Docker with NGINX + php7.4-fpm-alpine. To change the max file size you can upload to wordpress, you will need to make changes to two places:

NGINX

In the server block set the client_max_body_size like so

server { 
    listen 443 ssl; 
    server_name '"$COMMON_NAME"'; 
    ssl_certificate /home/server.crt;
    ssl_certificate_key /home/server.key;
    root /var/www/html; 
    client_max_body_size 100M;
  

This setting controls how much the web server itself will allow before wordpress even comes into the picture.

php7.4-fpm-alpine

Create following file if it does not exist /usr/local/etc/php/conf.d/uploads.ini and edit it like so:

file_uploads = On
memory_limit = 500M
upload_max_filesize = 100M
post_max_size = 100M
max_execution_time = 600
max_file_uploads = 50000
max_execution_time = 5000
max_input_time = 5000

You are all set! Above settings control how much wordpress will allow. Take a look at wp_max_upload_size

Tip: Verify /usr/local/etc/php/conf.d/uploads.ini is indeed the correct file to modify by running phpinfo() in PHP REPL

bash-5.0# php -a
Interactive shell

php > phpinfo();
phpinfo()
PHP Version => 7.4.5

...

Configuration File (php.ini) Path => /usr/local/etc/php
Loaded Configuration File => (none)
Scan this dir for additional .ini files => /usr/local/etc/php/conf.d
Additional .ini files parsed => /usr/local/etc/php/conf.d/docker-php-ext-bcmath.ini,
/usr/local/etc/php/conf.d/docker-php-ext-exif.ini,
/usr/local/etc/php/conf.d/docker-php-ext-gd.ini,
/usr/local/etc/php/conf.d/docker-php-ext-imagick.ini,
/usr/local/etc/php/conf.d/docker-php-ext-mysqli.ini,
/usr/local/etc/php/conf.d/docker-php-ext-opcache.ini,
/usr/local/etc/php/conf.d/docker-php-ext-sodium.ini,
/usr/local/etc/php/conf.d/docker-php-ext-zip.ini,
/usr/local/etc/php/conf.d/error-logging.ini,
/usr/local/etc/php/conf.d/opcache-recommended.ini,
/usr/local/etc/php/conf.d/uploads.ini