As a developer, you've probably at least heard something about CI - Continuous integration. And if you haven't - you better fix it ASAP, because that's something awesome to have on your skill list and can get extremely helpful in your everyday work. This post will focus on CI for PHP devs, and specifically, on CI implementation from Gitlab. I will suppose you know the basics of Git, PHP, PHPUnit, Docker and unix shell. Intended audience - intermediate PHP devs.
Adding something to your workflow must serve a purpose. In this case the goal is to automate routine tasks and achieve better quality control. Even a basic PHP project IMO needs the following:
* linter) checks (cannot merge changes that are invalid on the syntax level)
* Code style checks
* Unit and integration tests
All of those can be just run eventually, of course. But I prefer an automated CI approach even in my personal projects because it leads to a higher level of discipline, you simply can't avoid following a set of rules that you've developed. Also, it reduces a risk of releasing a bug or regression, thus improving quality.
Gitlab is as generous as giving you their CI for free, even for your private repos. At this point it is starting to look as advertising, therefore a quick comparison table for Gitlab, Github, Bitbucket. AFAIK, Github does not have a built-in solution, instead it is easily integrated with third parties, of which Travis CI seems to be the most popular - I will therefore mention Travis here.

Public repositories (OSS projects). All 3 providers have a free offer for the open-source community!

| Provider | Limits |
|---|---|
| Gitlab | 2,000 CI pipeline minutes per group per month, shared runners |
| Travis | Apparently unlimited |
| Bitbucket| 50 min/month, max 5 users, File storage <= 1Gb/month |

Private repositories

| Provider | Price | Limits |
|---|---|---|
| Gitlab | Free | 2,000 CI pipeline minutes per group per month, shared runners |
| Travis | $69/month | Unlimited builds, 1 job at a time |
| Bitbucket| Free | 50 min/month, max 5 users, File storage <= 1Gb/month |

Getting started

I made a small project based on Laravel framework and called it "ci-showcase". I work in Linux environment, and the commands I use in the examples, are for linux shell. They should be pretty much the same on Mac and nearly the same on Windows though.

composer create-project laravel/laravel ci-showcase

Next, I went to gitlab website and created a new public project: https://gitlab.com/crocodile2u/ci-showcase. Cloned the repo and copied all files and folders from the newly created project - the the new git repo. In the root folder, I placed a .gitignore file:

.idea
vendor
.env

Then the .env file:

APP_ENV=development

Then I generated the application encryption key: php artisan key:generate, and then I wanted to verify that the primary setup works as expected: ./vendor/bin/phpunit, which produced the output OK (2 tests, 2 assertions). Nice, time to commit this: git commit && git push

At this point, we don't yet have any CI, let's do something about it!

Adding .gitlab-ci.yml

Everyone going to implement CI with Gitlab, is strongly encouraged to bookmark this page: https://docs.gitlab.com/ee/ci/README.html. I will simply provide a short introduction course here plus a bit of boilerplate code to get you started easier.
First QA check that we're going to add is PHP syntax check. PHP has a built-in linter, which you can invoke like this: php -l my-file.php. This is what we're going to use. Because the php -l command doesn't support multiple files as arguments, I've written a small wrapper shell script and saved it to ci/linter.sh:

#!/bin/sh
files=<code>sh ci/get-changed-php-files.sh | xargs</code>last_status=0
status=0
# Loop through changed PHP files and run php -l on each
for f in "$files" ; do message=<code>php -l $f</code> last_status="$?" if [ "$last_status" -ne "0" ]; then # Anything fails -> the whole thing fails echo "PHP Linter is not happy about $f: $message" status="$last_status" fi
done
if [ "$status" -ne "0" ]; then echo "PHP syntax validation failed!"
fi
exit $status

Most of the time, you don't actually want to check each and every PHP file that you have. Instead, it's better to check only those files that have been changed. The Gitlab pipeline runs on every push to the repository, and there is a way to know which PHP files have been changed. Here's a simple script, meet ci/get-changed-php-files.sh:

#!/bin/sh
# What's happening here?
#
# 1. We get names and statuses of files that differ in current branch from their state in origin/master.
# These come in form (multiline)
# 2. The output from git diff is filtered by unix grep utility, we only need files with names ending in .php
# 3. One more filter: filter *out* (grep -v) all lines starting with R or D.
# D means "deleted", R means "renamed"
# 4. The filtered status-name list is passed on to awk command, which is instructed to take only the 2nd part
# of every line, thus just the filename
git diff --name-status origin/master | grep '\.php$' | grep -v "^[RD]" | awk '{ print }'

These scripts can easily be tested in your local environment ( at least if you have a Linux machine, that is ;-) ).
Now, as we have our first check, we'll finally create our .gitlab-ci.yml. This is where your pipeline is declared using YAML notation:

# we're using this beautiful tool for our pipeline: https://github.com/jakzal/phpqa
image: jakzal/phpqa:alpine
# For this sample pipeline, we'll only have 1 stage, in real-world you would like to also add at least "deploy"
stages: - QA
linter:
stage: QA
# this is the main part: what is actually executed
script:	- sh ci/get-changed-php-files.sh | xargs sh ci/linter.sh

The first line is image: jakzal/phpqa:alpine and it's telling Gitlab that we want to run our pipeline using a PHP-QA utility by jakzal. It is a docker image containing PHP and a huge variety of QA-tools. We declare one stage - QA, and this stage by now has just a single job named linter. Every job can have it's own docker image, but we don't need that for the purpose of this tutorial. Our project reaches Step 2. Once I had pushed these changes, I immediately went to the project's CI/CD page. Aaaand.... the pipeline was already running! I clicked on the linter job and saw the following happy green output:

Running with gitlab-runner 11.9.0-rc2 (227934c0) on docker-auto-scale ed2dce3a
Using Docker executor with image jakzal/phpqa:alpine ...
Pulling docker image jakzal/phpqa:alpine ...
Using docker image sha256:12bab06185e59387a4bf9f6054e0de9e0d5394ef6400718332c272be8956218f for jakzal/phpqa:alpine ...
Running on runner-ed2dce3a-project-11318734-concurrent-0 via runner-ed2dce3a-srm-1552606379-07370f92...
Initialized empty Git repository in /builds/crocodile2u/ci-showcase/.git/
Fetching changes...
Created fresh repository.
From https://gitlab.com/crocodile2u/ci-showcase * [new branch] master -> origin/master * [new branch] step-1 -> origin/step-1 * [new branch] step-2 -> origin/step-2
Checking out 1651a4e3 as step-2...
Skipping Git submodules setup
$ sh ci/get-changed-php-files.sh | xargs sh ci/linter.sh
Job succeeded

It means that our pipeline was successfully created and run!

PHP Code Sniffer.

PHP Code Sniffer is a tool for keeping app of your PHP files in one uniform code style. It has a hell of customizations and settings, but here we will only perform simple check for compatibilty with PSR-2 standard. A good practice is to create a configuration XML file in your project. I will put it in the root folder. Code sniffer can use a few file names, of which I prefer phpcs.xml:

<?xml version="1.0"?>
/resources

I also will append another section to .gitlab-ci.yml:

code-style:	stage: QA	script:	# Variable $files will contain the list of PHP files that have changes	- files=<code>sh ci/get-changed-php-files.sh</code> # If this list is not empty, we execute the phpcs command on all of them - if [ ! -z "$files" ]; then echo $files | xargs phpcs; fi

Again, we check only those PHP files that differ from master branch, and pass their names to phpcs utility. That's it, Step 3 is finished! If you go to see the pipeline now, you will notice that linter and code-style jobs run in parallel.

Adding PHPUnit

Unit and integration tests are essential for a successful and maintaiable modern software project. In PHP world, PHPUnit is de facto standard for these purposes. The PHPQA docker image already has PHPUnit, but that's not enough. Our project is based on Laravel, which means it depends on a bunch of third-party libraries, Laravel itself being one of them. Those are installed into vendor folder with composer. You might have noticed that our .gitignore file has vendor folder as one of it entries, which means that it is not managed by the Version Control System. Some prefer their dependencies to be part of their Git repository, I prefer to have only the composer.json declarations in Git. Makes the repo much much smaller than the other way round, also makes it easy to avoid bloating your production builds with libraries only needed for development.
Composer is also included into PHPQA docker image, and we can enrich our .gitlab-ci.yml:

test:	stage: QA	cache:	key: dependencies-including-dev	paths: - vendor/	script:	- composer install	- ./vendor/bin/phpunit

PHPUnit requires some configuration, but in the very beginning we used composer create-project to create our project boilerplate.laravel/laravel package has a lot of things included in it, and phpunit.xml is also one of them. All I had to do was to add another line to it:

xml

APP_KEY enironment variable is essential for Laravel to run, so I generated a key with php artisan key:generate.
git commit & git push, and we have all three jobs on theQA stage!

Checking that our checks work

In this branch I intentionally added changes that should fail all three job in our pipeline, take a look at git diff. And we have this out from the pipeline stages:Linter:

$ ci/linter.sh
PHP Linter is not happy about app/User.php:
Parse error: syntax error, unexpected 'syntax' (T_STRING), expecting function (T_FUNCTION) or const (T_CONST) in app/User.php on line 11
Errors parsing app/User.php
PHP syntax validation failed!
ERROR: Job failed: exit code 255

**Code-style**:

$ if [ ! -z "$files" ]; then echo $files | xargs phpcs; fi
FILE: ...ilds/crocodile2u/ci-showcase/app/Http/Controllers/Controller.php
----------------------------------------------------------------------
FOUND 0 ERRORS AND 1 WARNING AFFECTING 1 LINE
---------------------------------------------------------------------- 13 | WARNING | Line exceeds 120 characters; contains 129 characters
----------------------------------------------------------------------
Time: 39ms; Memory: 6MB
ERROR: Job failed: exit code 123

**test**:

$ ./vendor/bin/phpunit
PHPUnit 7.5.6 by Sebastian Bergmann and contributors.
F. 2 / 2 (100%)
Time: 102 ms, Memory: 14.00 MB
There was 1 failure:
1) Tests\Unit\ExampleTest::testBasicTest
This test is now failing
Failed asserting that false is true.
/builds/crocodile2u/ci-showcase/tests/Unit/ExampleTest.php:17
FAILURES!
Tests: 2, Assertions: 2, Failures: 1.
ERROR: Job failed: exit code 1

Congratulations, our pipeline is running, and we now have much less chance of messing up the result of our work.

Conclusion

Now you know how to set up a basic QA pipeline for your PHP project. There's still a lot to learn. Pipeline is a powerful tool. For instance, it can make deployments to different environments for you. Or it can build docker images, store artifacts and more! Sounds cool? Then spend 5 minutes of your time and leave a comment, you can also tell me if there is a pipeline topic you would like to be covered in next posts.

Hey Friends,

I am sharing a very interesting blog on how to create a chat system in php without using ajax. As we all know ajax based chat system in php is not a good solution
because itincreases the server load and redundant xhr calls on our server.

Instead, I am going to use sockets for incoming messages from and send messages to another user. So lets try them out using the following steps:

Step 1: Cross check in php.ini that sockets extension is enabled

;extension=sockets
extension=sockets

Step 2: Create server.php file

This file will handle the incoming and outgoing messages on sockets, Add following variables in top of the file:

$host = 'localhost';
$port = '9000';
$null = NULL; 

Step 3: After it add helper methods

The following code for handshake with new incoming connections and encrypt and decrypt messages incoming and outgoing over sockets:

function send_message($msg)
{
global $clients;
foreach($clients as $changed_socket)
{
@socket_write($changed_socket,$msg,strlen($msg));
}
return true;
}
function unmask($text) {
$length = ord($text[1]) & 127;
if($length == 126) {
$masks = substr($text, 4, 4);
$data = substr($text, 8);
}
elseif($length == 127) {
$masks = substr($text, 10, 4);
$data = substr($text, 14);
}
else {
$masks = substr($text, 2, 4);
$data = substr($text, 6);
}
$text = "";
for ($i = 0; $i < strlen($data); ++$i) {
$text .= $data[$i] ^ $masks[$i%4];
}
return $text;
}
function mask($text)
{
$b1 = 0x80 | (0x1 & 0x0f);
$length = strlen($text);

if($length <= 125)
$header = pack('CC', $b1, $length);
elseif($length > 125 && $length < 65536)
$header = pack('CCn', $b1, 126, $length);
elseif($length >= 65536)
$header = pack('CCNN', $b1, 127, $length);
return $header.$text;
}
function perform_handshaking($receved_header,$client_conn, $host, $port)
{
$headers = array();
$lines = preg_split("/

/", $receved_header);
foreach($lines as $line)
{
$line = chop($line);
if(preg_match('/\A(\S+): (.*)\z/', $line, $matches))
{
$headers[$matches[1]] = $matches[2];
}
}
$secKey = $headers['Sec-WebSocket-Key'];
$secAccept = base64_encode(pack('H*', sha1($secKey . '258EAFA5-E914-47DA-95CA-C5AB0DC85B11')));
$upgrade = "HTTP/1.1 101 Web Socket Protocol Handshake

" .
"Upgrade: websocket

" .
"Connection: Upgrade

" .
"WebSocket-Origin: $host

" .
"WebSocket-Location: ws://$host:$port/php-ws/chat-daemon.php

".
"Sec-WebSocket-Accept:$secAccept



";
socket_write($client_conn,$upgrade,strlen($upgrade));
}

Step 4: Now add following code to create bind and listen tcp/ip sockets:

$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
socket_set_option($socket, SOL_SOCKET, SO_REUSEADDR, 1);
socket_bind($socket, 0, $port);
socket_listen($socket);
$clients = array($socket);

Ok now a endless loop that will use for handeling incominga nd send messages:

while (true) {
$changed = $clients;
socket_select($changed, $null, $null, 0, 10);

if (in_array($socket, $changed)) {
$socket_new = socket_accept($socket);	$clients[] = $socket_new; 
$header = socket_read($socket_new, 1024);	perform_handshaking($header, $socket_new, $host, $port); 
socket_getpeername($socket_new, $ip);	$response = mask(json_encode(array('type'=>'system', 'message'=>$ip.' connected')));	send_message($response); 
$found_socket = array_search($socket, $changed);
unset($changed[$found_socket]);
}

foreach ($changed as $changed_socket) {

while(socket_recv($changed_socket, $buf, 1024, 0) >= 1)
{
$received_text = unmask($buf);	$tst_msg = json_decode($received_text, true);	$user_name = $tst_msg['name'];	$user_message = $tst_msg['message'];	$user_color = $tst_msg['color']; 
$response_text = mask(json_encode(array('type'=>'usermsg', 'name'=>$user_name, 'message'=>$user_message, 'color'=>$user_color)));
send_message($response_text);	break 2;	}

$buf = @socket_read($changed_socket, 1024, PHP_NORMAL_READ);
if ($buf === false) {	$found_socket = array_search($changed_socket, $clients);
socket_getpeername($changed_socket, $ip);
unset($clients[$found_socket]);

$response = mask(json_encode(array('type'=>'system', 'message'=>$ip.' disconnected')));
send_message($response);
}
}
}
socket_close($socket);

So you are ready with server side socket program, Now its time to move on front side where we will implement w3c provided client side Web Socket Apis,

Step 5: create a file named index.php for frontend usage with following initial code

$host = 'localhost';
$port = '9000';
$subfolder = "php_ws/";
$colors = array('#007AFF','#FF7000','#FF7000','#15E25F','#CFC700','#CFC700','#CF1100','#CF00BE','#F00');
$color_pick = array_rand($colors);
<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<div class="chat-wrapper">
<div id="message-box"></div>
<div class="user-panel">
<input type="text" name="name" id="name" placeholder="Your Name" maxlength="15" />
<input type="text" name="message" id="message" placeholder="Type your message here..." maxlength="100" />
<button id="send-message">Send</button>
</div>
</div>
</body>
</html>

Now add some basic styling in the head section using following code:

<style type="text/css">
.chat-wrapper {
font: bold 11px/normal 'lucida grande', tahoma, verdana, arial, sans-serif;
 background: #00a6bb;
 padding: 20px;
 margin: 20px auto;
 box-shadow: 2px 2px 2px 0px #00000017;
max-width:700px;
min-width:500px;
}
#message-box {
 width: 97%;
 display: inline-block;
 height: 300px;
 background: #fff;
 box-shadow: inset 0px 0px 2px #00000017;
 overflow: auto;
 padding: 10px;
}
.user-panel{
 margin-top: 10px;
}
input[type=text]{
 border: none;
 padding: 5px 5px;
 box-shadow: 2px 2px 2px #0000001c;
}
input[type=text]#name{
 width:20%;
}
input[type=text]#message{
 width:60%;
}
button#send-message {
 border: none;
 padding: 5px 15px;
 background: #11e0fb;
 box-shadow: 2px 2px 2px #0000001c;
}
</style>

Ok Style is all set now need to add a jquery script and create web socket object and handle all events on it as following code need to add before closing of bosy tag:

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<script language="javascript" type="text/javascript"> 
var msgBox = $('#message-box');
var wsUri = "ws://".$host.":".$port."/php-ws/server.php"; 
websocket = new WebSocket(wsUri); 

websocket.onopen = function(ev) {	msgBox.append('<div class="system_msg" style="color:#bbbbbb">Welcome to my "Chat box"!</div>');	}
websocket.onmessage = function(ev) {
var response	= JSON.parse(ev.data); 
var res_type	= response.type;	var user_message	= response.message;	var user_name	= response.name;	var user_color	= response.color;	switch(res_type){
case 'usermsg':
msgBox.append('<div><span class="user_name" style="color:' + user_color + '">' + user_name + '</span> : <span class="user_message">' + user_message + '</span></div>');
break;
case 'system':
msgBox.append('<div style="color:#bbbbbb">' + user_message + '</div>');
break;
}
msgBox[0].scrollTop = msgBox[0].scrollHeight;	};

websocket.onerror	= function(ev){ msgBox.append('<div class="system_error">Error Occurred - ' + ev.data + '</div>'); }; 
websocket.onclose	= function(ev){ msgBox.append('<div class="system_msg">Connection Closed</div>'); }; 
$('#send-message').click(function(){
send_message();
});

$( "#message" ).on( "keydown", function( event ) {
 if(event.which==13){
 send_message();
 }
});

function send_message(){
var message_input = $('#message');	var name_input = $('#name'); 
if(message_input.val() == ""){	alert("Enter your Name please!");
return;
}
if(message_input.val() == ""){	alert("Enter Some message Please!");
return;
}
var msg = {
message: message_input.val(),
name: name_input.val(),
color : '<?php echo $colors[$color_pick]; ?>'
};
websocket.send(JSON.stringify(msg));
message_input.val('');	}
</script>

Ok All set, Now need to run the server.php file using following php-cli utility,make sure you have php cli utility installed in your system:


php -q c:\xampp\htdocs\php-ws\server.php

Now you may access the front index.php file via the browser url like following and see a chatbox and connection status, you may use the same url or different browser to check the chat system is working or not.

PHPUnit is an essential tool for every PHP developers. It is one of those tools that every PHP developer should have installed in their development environment. The problems most first time PHPUnit developers run into are where to install it and how to install it. This quick guide will walk you through the process and answer both questions.

How do I install PHPUnit

The Easy Way

In your project’s root directory use this command.

composer require --dev phpunit/phpunit ^6.0

This command adds PHPUnit to your project as a development dependency. This is the absolute best way to install PHPUnit. It is the best way because this way the version of PHPUnit does not change unless you change it. We specified ^6.0 as the version which means we’ll get all the updates to the 6.0 branch but not 6.1. While BC breaks don’t happen often in PHPUnit, they have happened. If you have a globally installed version of PHPUnit and you upgrade it to a version that breaks BC, you have to go update all of your tests immediately. This is a lot of hassle if you have a lot of projects with a lot of tests. Keeping a copy of PHPUnit installed as a dev requirement in each project means that each project has its own copy that can be upgraded as necessary.

The Hard Way

In a command prompt regardless of where you are in your file system, use this command.

composer global require phpunit/phpunit ^6.0

On MacOS and Linux machines, this will install PHPUnit in ~/.composer/vendor/bin. If you add this directory to your path, then from any project, you can execute PHPUnit. However, as noted above, if you ever upgrade your globally installed packages then you will have problems.

composer global update

Run that when there is a new version of PHPUnit, it will be installed, regardless of whether this will break your existing unit tests on one or more of your projects. Windows users will need to locate the .composer/vendor/bin directory in your user’s home directory.

The “ZOMG why would you do it this way” Way

Here is the old-school use wget and move it into the correct position manually. You can do it this way, but you will have to take care of all upgrades manually as well. If you only have a single project on the computer and you never ever plan on changing the version of PHPUnit….nope, still better to usecomposer require --dev.

wget https://phar.phpunit.de/phpunit-6.0.phar

chmod +x phpunit-6.0.phar

sudo mv phpunit-6.0.phar /usr/local/bin/phpunit

phpunit --version

These instructions are of course for MacOS or Linux. Windows user won’t need to do chmod or sudo but will need a BAT file.

That’s it. One of those commands should get you a working copy of PHPUnit on your computer.

Resources:

* Installing PHPUnit
* Composer Introduction (For the Global option)

In his talk, “Writing Viruses for Fun, Not Profit,”Ben Dechrai (after making the viewer take a pledge to only use this knowledge for good and not evil) walks through how many viruses operate, and just how easy it is to build your own self-replicating virus in PHP.

The danger of many of these viruses according to Ben is that the most dangerous viruses often escape detection by not looking like a virus. Instead they encrypt their code to hide their true intent, while also constantly adapting and evolving.

Perhaps even more dangerously, they act like they’re benign and don’t actually do anything - often times laying dormant until called upon by the malicious actor.

Creating the Virus

What’s scary is just how simple it was for Ben to create such a virus, one that mutated ever so slightly as it infected every other file on the server. Opening up unlimited possibilities from scraping customer data, to DDOS attacks, to simply hijacking your domain.



But those attacks are just the start as Ben demonstrated how easy it is to write new files, delete files, eval() and execute foreign code - which could even be extended to accessing the underlying server itself if shell_exec() is enabled.

To add to the problem, Ben shares how challenging it can be to identify malicious code on your server as many of these attacks are far more sophisticated than the the virus he created in a matter of minutes - hiding themselves and often appearing as if they are part of the original source code.

Deploying the Virus

To drive his point home, Ben demonstrates how even seemingly secure systems can be vulnerable - as all it takes is one tiny misstep within your application.

He highlights this by building what should be a secure photo gallery - one that checks the extension and mime-type of the image - and even stores it outside of the public directory. He goes even farther by adding additional sanity checks with a PHP script that then renders the image.

After walking through the code and it’s security features, he then downloads a simple image from the internet. Opening his editor he quickly injects the virus (written in PHP) into the image and uploads it, passing all of the server checks.

Surely, since it passed these checks the system is secure, right? Ben loads the gallery to proudly show off the image - which is just that… an image, with nothing special or out of the ordinary.
Except that when he opens the image gallery files, each has been infected with the malicious code.

The culprit that allowed for Ben to hijack an entire system and execute foreign code, create new files, and even hijack the entire site? When displaying the image the file was included using PHP’s include() function, instead of pulling in the data using file_get_contents() and echoing it out.

Such a simple mistake provided Ben, if he was a malicious hacker, complete access to all of the files on the system.

Protecting Yourself

Security always exists in layers - and this could have been prevented by including a few more layers, such as using an open source library to rewrite the image, reviewing the image source before pulling it in, or again not giving it executable access by using the PHP include() function.

But what’s terrifying is how simple it is to hijack a site, how easy it is to get access to your system and private data, and how easy it is to overlook security vulnerabilities - especially with open source tooling and those that take plugins.

As Ben explains, sometimes the core code itself is really secure, but then you get two different plugins that when used together accidentally create a security vulnerability. That by itself is one of the most challenging as you can audit each plugin individually, and still not know you’re opening up your system to malicious actors.

This is why it's not just important to stay up to date on the latest security measures and best practices, but to be constantly thinking like a hacker and testing your code for vulnerabilities.

Learn More

You can watch thefull video to learn more how viruses operate, how to quickly build your own PHP virus (but you must promise to use it for good), and what to watch for in order to protect yourself, your customers, and your architecture.

Introduction

In a previous article we learned about Creating a PHP Daemon Service. Now we are going to learn how to use methods to perform IPC - Inter-Process Communication - to communicate with daemon processes.

Message Queues

In the world of UNIX, there is an incredible variety of ways to send a message or a command to a daemon script and vice versa. But first I want to talk only about message queues - "System V IPC Messages Queues".

A long time ago I learned that a queue can be either in the System V IPC implementation, or in the POSIX implementation. I want to comment only about the System V implementation, as I know it better.

Lets get started. At the "normal" operating system level, queues are stored in memory. Queue data structures are available to all system programs. Just as in the file system, it is possible to configure queues access rights and message size. Usually a queue message size is small, less than 8 KB.

This introductory part is over. Lets move on to the practice with same example scripts.queue-send.php

$key = ftok(__FILE__, 'A'); 
$queue = msg_get_queue($key);

msg_send($queue, 1, 'message, type 1');
msg_send($queue, 2, 'message, type 2');
msg_send($queue, 3, 'message, type 3');
msg_send($queue, 1, 'message, type 1');

echo "send 4 messages
";

queue-receive.php

$key = ftok('queue-send.php', 'A');
$queue = msg_get_queue($key);

for ($i = 1; $i <= 3; $i++) {
 echo "type: {$i}
";

 while ( msg_receive($queue, $i, $msgtype, 4096, $message, false, MSG_IPC_NOWAIT) ) {
 echo "type: {$i}, msgtype: {$msgtype}, message: {$message}
";
 }
}

Lets run on the first stage of the file queue-send.php, and then queue-receive.php.

u% php queue-send.php
send 4 messages
u% php queue-receive.php
type: 1
type: 1, msgtype: 1, message: s:15:"message, type 1";
type: 1, msgtype: 1, message: s:15:"message, type 1";
type: 2
type: 2, msgtype: 2, message: s:15:"message, type 2";
type: 3
type: 3, msgtype: 3, message: s:15:"message, type 3";

You may notice that the messages have been grouped. The first group gathered 2 messages of the first type, and then the remaining messages.

If we would have indicated to receive messages of type 0, you would get all messages, regardless of the type.

while (msg_receive($queue, $i, $msgtype, 4096, $message, false, MSG_IPC_NOWAIT)) {

Here it is worth noting another feature of the queues: if we do not use the constant MSG_IPC_NOWAIT in the script and run the script queue-receive.php from a terminal, and then run periodically the file queue-send.php, we see how a daemon can effectively use this to wait jobs.queue-receive-wait.php

$key = ftok('queue-send.php', 'A');
$queue = msg_get_queue($key);

while ( msg_receive($queue, 0, $msgtype, 4096, $message) ) {
 echo "msgtype: {$msgtype}, message: {$message}
";
}

Actually that is the most interesting information of all I have said. There are also functions to get statistics, disposal and checking for the existence of queues.

Lets now try to write a daemon listening to a queue:queue-daemon.php

$pid = pcntl_fork();
$key = ftok('queue-send.php', 'A');
$queue = msg_get_queue($key);

if ($pid == -1) {
 exit;
} elseif ($pid) {
 exit;
} else {
 while ( msg_receive($queue, 0, $msgtype, 4096, $message) ) {
 echo "msgtype: {$msgtype}, message: {$message}
";
 }
}

posix_setsid();

Shared Memory

We have learned to work with queues, with which you can send small system messages. But then we may certainly be faced with the task of transmitting large amounts of data. My favorite type of system, System V, has solved the problem of rapid transmission and preservation of large data in memory using a mechanism calledShared Memory.

In short, the data in the Shared Memory lives until the system is rebooted. Since the data is in memory, it works much faster than if it was stored in a database somewhere in a file, or, God forgive me on a network share.

Lets try to write a simple example of data storage.shared-memory-write-base.php

$id = ftok(__FILE__, 'A');


$shmId = shm_attach($id);

$var = 1;

if (shm_has_var($shmId, $var)) {
 $data = (array) shm_get_var($shmId, $var);
} else {
 $data = array();
}

$data[time()] = file_get_contents(__FILE__);

shm_put_var($shmId, $var, $data);

Run this script several times to save the value in memory. Now lets write a script only to read from the memory.shared-memory-read-base.php

$id = ftok(__DIR__ . '/shared-memory-write-base.php', 'A');
$shmId = shm_attach($id);
$var = 1; 

if (shm_has_var($shmId, $var)) {
 $data = (array) shm_get_var($shmId, $var);
} else {
 $data = array();
}

foreach ($data as $key => $value) {
 $path = "/tmp/$key.php";
 file_put_contents($path, $value);

 echo $path . PHP_EOL;
} 

Semaphores

So, in general terms, it should be clear for you by now how to work with shared memory. The only problems left to figure out are about a couple of nuances, such as: "What to do if two processes want to record one block of memory?" Or "How to store binary files of any size?".

To prevent simultaneous accesses we will use semaphores. Semaphores allow us to flag that we want to have exclusive access to some resource, like for instance a shared memory block. While that happens other processes will wait for their turn on semaphore.

In this code it explained clearly:shared-memory-semaphors.php

$id = ftok(__FILE__, 'A');

$semId = sem_get($id);

sem_acquire($semId);

$data = file_get_contents(__DIR__.'/06050396.JPG', FILE_BINARY);

$shmId = shm_attach($id, strlen($data)+4096);
$var = 1;
 
if (shm_has_var($shmId, $var)) {
 $data = shm_get_var($shmId, $var);

 $filename = '/tmp/' . time();
 file_put_contents($filename, $data, FILE_BINARY);

 shm_remove($shmId);
} else {
 shm_put_var($shmId, $var, $data); 
}

sem_release($semId); 

Now you can use the md5sum command line utility to compare two files, the original and the saved file. Or, you can open the file in image editor or whatever prefer to compare the images.

With this we are done with shared memory and semaphores. As your homework I want to ask you to write code that a demon will use semaphores to access shared memory.

Conclusion

Exchanging data between the daemons is very simple. This article described two options for data exchange: message queues and shared memory.

Post a comment here if you have questions or comments about how to exchange data with daemon services in PHP.