Models and controllers are one of the most essential programming handlers in the Laravel MVC framework, and both are used vastly for different functional operations. Models in Laravel are created inside the app folder and are mostly used to interact with the database using Eloquent ORM, while the controllers are located inside the directory App/Http/Controllers.

As a programmer, you should have the knowledge how to keep the balance in between the programming usage of Models and controllers. As which one should be more utilized for allowing functional tasks in applications deployed on any PHP MySQL hosting.

What is the Concept of Thin Controller and FAT Models

The concept of the thin controller and fat model is that we do less work in our controllers and more work in our models. Like we use our controllers to validate our data and then pass it to the models. While in models, we define our actual functional logic and main coding operations of the desired application. This code structuring process is also a very basic concept of MVC and also the differentiating factor from the conventional complex programming which we mistakenly ignore sometimes.

Why FAT Controllers Are Bad For Handling Code

Controllers are always meant to be defined short and concise, and it should only be used for receiving requests and return responses to it. Anything else further should be programmed in Models, which is actually made for main functional operations.

Placing functional logic in controllers can be bad for many reasons for your applications deployed on anyhosting for PHP. As it not only makes code structure long but also makes it complex sometimes. Further placing code in Controllers is also not recommended because if same functionality is needed somewhere else in route, then pulling out the whole code from their becomes difficult and so its reusability in the application.

Though Laravel is an MVC framework while developing on laravel, we sometimes ignore this and write mostly all our code including the extending of App\Model and all our functional logic in controller route methods. What we can do here is we can create a sub model of our parent model. For example, our parent model is User then we can create another sub model of username in CustomerModel if you are using the same User model for all types of users. In this model, we will write all the logic related to user type Customer.

So now let's take an example of my existing blog creating comment system with laravel and vuejs. In that article, you can see I have made so much mess in my controller methods. Mostly, I have written all my comments logic in my methods, so to shorten that let's clean them in this article. Inside app folder, I will create a new file with name CommentModel.php. Inside this file, I will write my whole logic for comment functions. This is my basic file:

<?php
namespace App;

use App\Comment;
use App\CommentVote;
use App\CommentSpam;
use App\User;
use Auth;

class CommentModel
{


}

?>

Right now it contains no function but has the reference of all my models which I required for this model. Let's first add a function namedgetallcomments passing$pageId as a parameter inside it. The function will get all the comments for the given page:

public function getAllComments($pageId)
 {
 $comments = Comment::where('page_id',$pageId)->get();

 $commentsData = [];


 foreach ($comments as $key) {
 $user = User::find($key->users_id);
 $name = $user->name;
 $replies = $this->replies($key->id);
 $photo = $user->first()->photo_url;
 $reply = 0;
 $vote = 0;
 $voteStatus = 0;
 $spam = 0;
 if(Auth::user()){
 $voteByUser = CommentVote::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();
 $spamComment = CommentSpam::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();


 if($voteByUser){
 $vote = 1;
 $voteStatus = $voteByUser->vote;
 }

 if($spamComment){
 $spam = 1;
 }
 }


 if(sizeof($replies) > 0){
 $reply = 1;
 }

 if(!$spam){
 array_push($commentsData,[
 "name" => $name,
 "photo_url" => (string)$photo,
 "commentid" => $key->id,
 "comment" => $key->comment,
 "votes" => $key->votes,
 "reply" => $reply,
 "votedByUser" =>$vote,
 "vote" =>$voteStatus,
 "spam" => $spam,
 "replies" => $replies,
 "date" => $key->created_at->toDateTimeString()
 ]);
 } 


 }
 $collection = collect($commentsData);
 return $collection->sortBy('votes');
 }

Now I will create another function namedreplies which takes$commentId as a parameter. The function is more or less programmed in the same manner as the upper function get all comments.

protected function replies($commentId)
 {
 $comments = Comment::where('reply_id',$commentId)->get();
 $replies = [];



 foreach ($comments as $key) {
 $user = User::find($key->users_id);
 $name = $user->name;
 $photo = $user->first()->photo_url;

 $vote = 0;
 $voteStatus = 0;
 $spam = 0; 


 if(Auth::user()){
 $voteByUser = CommentVote::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();
 $spamComment = CommentSpam::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();

 if($voteByUser){
 $vote = 1;
 $voteStatus = $voteByUser->vote;
 }

 if($spamComment){
 $spam = 1;
 }
 }
 if(!$spam){


 array_push($replies,[
 "name" => $name,
 "photo_url" => $photo,
 "commentid" => $key->id,
 "comment" => $key->comment,
 "votes" => $key->votes,
 "votedByUser" => $vote,
 "vote" => $voteStatus,
 "spam" => $spam,
 "date" => $key->created_at->toDateTimeString()
 ]);
 }




 }


 $collection = collect($replies);
 return $collection->sortBy('votes');
 }

Now lets create a functioncreate comment which passes$array as a parameter in it:

public function createComment($arary)
 {
 $comment = Comment::create($array);


 if($comment)
 return [ "status" => "true","commentId" => $comment->id ];
 else
 return [ "status" => "false" ]; 
 }

Similarly, Now I will create all the function for comment in myCommentModel, so that all the functions gets accumulated in one model.

<?php
namespace App;

use App\Comment;
use App\CommentSpam;
use App\CommentVote;
use App\User;
use Auth;

class CommentModel
{
 public function getAllComments($pageId)
 {
 $comments = Comment::where('page_id', $pageId)->get();

 $commentsData = [];

 foreach ($comments as $key) {
 $user = User::find($key->users_id);
 $name = $user->name;
 $replies = $this->replies($key->id);
 $photo = $user->first()->photo_url;
 $reply = 0;
 $vote = 0;
 $voteStatus = 0;
 $spam = 0;
 if (Auth::user()) {
 $voteByUser = CommentVote::where('comment_id', $key->id)->where('user_id', Auth::user()->id)->first();
 $spamComment = CommentSpam::where('comment_id', $key->id)->where('user_id', Auth::user()->id)->first();

 if ($voteByUser) {
 $vote = 1;
 $voteStatus = $voteByUser->vote;
 }

 if ($spamComment) {
 $spam = 1;
 }
 }

 if (sizeof($replies) > 0) {
 $reply = 1;
 }

 if (!$spam) {
 array_push($commentsData, [
 "name" => $name,
 "photo_url" => (string) $photo,
 "commentid" => $key->id,
 "comment" => $key->comment,
 "votes" => $key->votes,
 "reply" => $reply,
 "votedByUser" => $vote,
 "vote" => $voteStatus,
 "spam" => $spam,
 "replies" => $replies,
 "date" => $key->created_at->toDateTimeString(),
 ]);
 }

 }
 $collection = collect($commentsData);
 return $collection->sortBy('votes');
 }

 protected function replies($commentId)
 {
 $comments = Comment::where('reply_id', $commentId)->get();
 $replies = [];

 foreach ($comments as $key) {
 $user = User::find($key->users_id);
 $name = $user->name;
 $photo = $user->first()->photo_url;

 $vote = 0;
 $voteStatus = 0;
 $spam = 0;

 if (Auth::user()) {
 $voteByUser = CommentVote::where('comment_id', $key->id)->where('user_id', Auth::user()->id)->first();
 $spamComment = CommentSpam::where('comment_id', $key->id)->where('user_id', Auth::user()->id)->first();

 if ($voteByUser) {
 $vote = 1;
 $voteStatus = $voteByUser->vote;
 }

 if ($spamComment) {
 $spam = 1;
 }
 }
 if (!$spam) {

 array_push($replies, [
 "name" => $name,
 "photo_url" => $photo,
 "commentid" => $key->id,
 "comment" => $key->comment,
 "votes" => $key->votes,
 "votedByUser" => $vote,
 "vote" => $voteStatus,
 "spam" => $spam,
 "date" => $key->created_at->toDateTimeString(),
 ]);
 }

 }

 $collection = collect($replies);
 return $collection->sortBy('votes');
 }

 public function createComment($arary)
 {
 $comment = Comment::create($array);

 if ($comment) {
 return ["status" => "true", "commentId" => $comment->id];
 } else {
 return ["status" => "false"];
 }

 }

 public function voteComment($commentId, $array)
 {
 $comments = Comment::find($commentId);
 $data = [
 "comment_id" => $commentId,
 'vote' => $array->vote,
 'user_id' => $array->users_id,
 ];

 if ($array->vote == "up") {
 $comment = $comments->first();
 $vote = $comment->votes;
 $vote++;
 $comments->votes = $vote;
 $comments->save();
 }

 if ($array->vote == "down") {
 $comment = $comments->first();
 $vote = $comment->votes;
 $vote--;
 $comments->votes = $vote;
 $comments->save();
 }

 if (CommentVote::create($data)) {
 return true;
 }

 }

 public function spamComment($commentId, $array)
 {
 $comments = Comment::find($commentId);

 $comment = $comments->first();
 $spam = $comment->spam;
 $spam++;
 $comments->spam = $spam;
 $comments->save();

 $data = [
 "comment_id" => $commentId,
 'user_id' => $array->users_id,
 ];

 if (CommentSpam::create($data)) {
 return true;
 }

 }
}
?>

Now we have all our required methods inCommentModel. So now let's clean upCommentController which is currently bit complex and lengthy in code structure. As right nowCommentController look like this:

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use App\Http\Requests;
use App\Comment;
use App\CommentVote;
use App\CommentSpam;
use App\User;
use Auth;

class CommentController extends Controller
{


 
 public function index($pageId)
 {
 $comments = Comment::where('page_id',$pageId)->get();

 $commentsData = [];




 foreach ($comments as $key) {
 $user = User::find($key->users_id);
 $name = $user->name;
 $replies = $this->replies($key->id);
 $photo = $user->first()->photo_url;
 $reply = 0;
 $vote = 0;
 $voteStatus = 0;
 $spam = 0;
 if(Auth::user()){
 $voteByUser = CommentVote::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();
 $spamComment = CommentSpam::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();


 if($voteByUser){
 $vote = 1;
 $voteStatus = $voteByUser->vote;
 }

 if($spamComment){
 $spam = 1;
 }
 }


 if(sizeof($replies) > 0){
 $reply = 1;
 }

 if(!$spam){
 array_push($commentsData,[
 "name" => $name,
 "photo_url" => (string)$photo,
 "commentid" => $key->id,
 "comment" => $key->comment,
 "votes" => $key->votes,
 "reply" => $reply,
 "votedByUser" =>$vote,
 "vote" =>$voteStatus,
 "spam" => $spam,
 "replies" => $replies,
 "date" => $key->created_at->toDateTimeString()
 ]);
 } 


 }
 $collection = collect($commentsData);
 return $collection->sortBy('votes');
 }

 protected function replies($commentId)
 {
 $comments = Comment::where('reply_id',$commentId)->get();
 $replies = [];



 foreach ($comments as $key) {
 $user = User::find($key->users_id);
 $name = $user->name;
 $photo = $user->first()->photo_url;

 $vote = 0;
 $voteStatus = 0;
 $spam = 0; 


 if(Auth::user()){
 $voteByUser = CommentVote::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();
 $spamComment = CommentSpam::where('comment_id',$key->id)->where('user_id',Auth::user()->id)->first();

 if($voteByUser){
 $vote = 1;
 $voteStatus = $voteByUser->vote;
 }

 if($spamComment){
 $spam = 1;
 }
 }
 if(!$spam){


 array_push($replies,[
 "name" => $name,
 "photo_url" => $photo,
 "commentid" => $key->id,
 "comment" => $key->comment,
 "votes" => $key->votes,
 "votedByUser" => $vote,
 "vote" => $voteStatus,
 "spam" => $spam,
 "date" => $key->created_at->toDateTimeString()
 ]);
 }




 }


 $collection = collect($replies);
 return $collection->sortBy('votes');
 }

 
 public function store(Request $request)
 {
 $this->validate($request, [
 'comment' => 'required',
 'reply_id' => 'filled',
 'page_id' => 'filled',
 'users_id' => 'required',
 ]);
 $comment = Comment::create($request->all());
 if($comment)
 return [ "status" => "true","commentId" => $comment->id ];
 }

 
 public function update(Request $request, $commentId,$type)
 {
 if($type == "vote"){


 $this->validate($request, [
 'vote' => 'required',
 'users_id' => 'required',
 ]);

 $comments = Comment::find($commentId);
 $data = [
 "comment_id" => $commentId,
 'vote' => $request->vote,
 'user_id' => $request->users_id,
 ];

 if($request->vote == "up"){
 $comment = $comments->first();
 $vote = $comment->votes;
 $vote++;
 $comments->votes = $vote;
 $comments->save();
 }

 if($request->vote == "down"){
 $comment = $comments->first();
 $vote = $comment->votes;
 $vote--;
 $comments->votes = $vote;
 $comments->save();
 }

 if(CommentVote::create($data))
 return "true";
 }

 if($type == "spam"){


 $this->validate($request, [
 'users_id' => 'required',
 ]);

 $comments = Comment::find($commentId);


 $comment = $comments->first();
 $spam = $comment->spam;
 $spam++;
 $comments->spam = $spam;
 $comments->save();

 $data = [
 "comment_id" => $commentId,
 'user_id' => $request->users_id,
 ];

 if(CommentSpam::create($data))
 return "true";
 }
 }

 
 public function destroy($id)
 {
 }
}?>

After cleaning up the controller it will look much simpler and easy to understand like this:

<?php

namespace App\Http\Controllers;

use App\CommentModel;
use Illuminate\Http\Request;

class CommentController extends Controller
{

 private $commentModel = null;
 private function __construct()
 {
 $this->commentModel = new CommentModel();
 }

 
 public function index($pageId)
 {
 return $this->commentModel->getAllComments($pageId);
 }

 
 public function store(Request $request)
 {
 $this->validate($request, [
 'comment' => 'required',
 'reply_id' => 'filled',
 'page_id' => 'filled',
 'users_id' => 'required',
 ]);
 return $this->commentModel->createComment($request->all());
 }

 
 public function update(Request $request, $commentId, $type)
 {
 if ($type == "vote") {

 $this->validate($request, [
 'vote' => 'required',
 'users_id' => 'required',
 ]);

 return $this->commentModel->voteComment($commentId, $request->all());
 }

 if ($type == "spam") {

 $this->validate($request, [
 'users_id' => 'required',
 ]);

 return $this->commentModel->spamComment($commentId, $request->all());
 }
 }

}
?>

Wrap Up!

So Isn't it looking much cleaner and simpler to understand now? This is what actually a thin controller and fat model looks like. We have all our logic related to Comment system programmed in ourCommentModel and our controller is now just used to transfer data from the user to our model and returning the response which is coming from our model.

So this is how the structuring of the thin controller and fat model is made. Give your thoughts in the comments below.

Hey There,
I am expecting you are familiar with PHP. In this post I will be using the Laravel framework to create a small blog system. I am showing here very simple steps to create blogs, If you want this complete code then please message me.
What are major Prequisites for Laravel:
* PHP version >= 5.6
* Composer should be installed in system

Create a project with name tiny_blog with following command

composer create-project laravel/laravel --prefer-dist tiny_blog

enter into the laravel project

cd tiny_blog

create a migration file using following artisan command
<pre>php artisan make:migration create_blog_table</pre>
After this command you will found a new file created in database/migrations folder in your project, Just edit the file having 'create_blog_table' appended in its name

Now replace following code to create table schema with function up(), So now the method will look like following:

public function up()
{
 Schema::create('blogs', function (Blueprint $table) {
$table->increments('id');
$table->integer('user_id');
$table->string('category');
$table->string('title');
$table->text('description');
$table->timestamps();
});

}

replace following snippet with down method, it will look like following:

public function down()
 {
 Schema::dropIfExists('blogs');
 }

Its time to run the migration file we have created

php artisan migrate

After running,It will create the blogs table in database.Now time to create form and insert data into the table

Laravel itsef provide authentication , use following artisan command :

php artisan make:auth

Now start Larvel:

php artisan serve

it will start the laravel development server at http://127.0.0.1:8000

Now if you run that url the basic default ui will be created and login & register link you can see in Top right position of header

You can register and login now.this feature is provided by authentication module.
Now we need to create a controller for manage blogs with following command:

php artisan make:controller BlogController

will create a file namedBlogController.php in** app/HTTP/controllers** folder location

Now we need to create a Model also, use following command

php artisan make:model Blog

will create a file namedBlog.php in app folder location

Now in Controller we need to create a method for create blogs and available that method in Routes to access it via url. Just editroutes/web.php file and add the following line

Route::get('blog/create','BlogController@createBlog');

/create/blog/ will be url route that land on Blog Controller's createBlog method using get method.

Now before running this route just go to the app/Http/Controllers folder and Edit BlogController.php file and Add the createBlog method in that class as following

public function createBlog()
{
 return view('blog.create');
}

This code will try to load the view from/resources/views/blog/create.blade.php

In Laravel blade is a template engine. As we had not created the view file yet, so we need to create a blog folder inside/resources/views/ folder then inside blog folder create a file create.blade.php with following form

@extends('layouts.app')

@section('content')
<div class="container">
@if ($errors->any())
 <div class="alert alert-danger">
 <ul>
 @foreach ($errors->all() as $error)
 <li>{{ $error }}</li>
 @endforeach
 </ul>
 </div><br />
@endif
 <div class="row">
 <form method="post" action="{{url('blog/create')}}">
 <div class="form-group">
 <input type="hidden" value="{{csrf_token()}}" name="_token" />
 <label for="title">Title:</label>
 <input type="text" class="form-control" name="title"/>
 </div>
 <div class="form-group">
 <label for="title">Category/Tags:</label>
 <input type="text" class="form-control" name="category"/>
 </div>
 <div class="form-group">
 <label for="description">Description:</label>
 <textarea cols="10" rows="10" class="form-control" name="description"></textarea>
 </div>
 <button type="submit" class="btn btn-primary">Submit</button>
 </form>
 </div>
</div>
@endsection

Now we need to add a additional route to handle the post request on blog/create route, Just edit routes/web.php file and just add following line in last:

Route::post('blog/create','BlogController@saveBlog'); 

post route to handle the form post on route blog/create


Now create a method name saveBlog to save the user input data in the form

 public function saveBlog(Request $request)
 {
 $blog = new Blog();
 
 $this->validate($request, [
 'title'=>'required',
'category'=>'required',
 'description'=> 'required'
 ]);
 
 $blog->createBlog($request->all());
 return redirect('blog/index')->with('success', 'New blog has been created successfully :)'); }

Notice This method is using Blog object that we don't know that where it comes from? , So to make above code working we need to include the model which we created earlier need to include in our controller file So use following code to include it before the class created.

use App\Blog;

Now following line shows that there is a method named createBlog in Model(app/Blog.php), but in actual it is not there:

$blog->createBlog($data);

So go to the file app/Blog.php and Edit it and inside the class add following method:

 public function createBlog($data)
{

$this->user_id = auth()->user()->id;
$this->title = $data['title'];
$this->description = $data['description'];
$this->category = $data['category'];
$this->save();
return 1;
}

Now the creation of blog task has been done , Its time to show the created Entries So just create a route blog/index in routes/web.php

Route::get('blog/index','BlogController@showAllBlogs');

get route blog/index to show all the created blogs by current user


Now just add a method in controller

public function showAllBlogs()
 {
 $blogs = Blog::where('user_id', auth()->user()->id)->get();
 
 return view('blog.index',compact('blogs'));
 }

This method requires to create a index view in blog folder , So create a file named index.blade.php in /resources/views/blog/ folder with following code

@extends('layouts.app')

@section('content')
<div class="container">
 @if(\Session::has('success'))
 <div class="alert alert-success">
 {{\Session::get('success')}}
 </div>
 @endif
 <a type="button" href="{{url('blog/create')}}" class="btn btn-primary">Add New Blog</a>
 <br>
 <table class="table table-striped">
 <thead>
 <tr>
 <td>ID</td>
 <td>Title</td>
 <td>Category</td>
 <td>Description</td>
 <td colspan="2">Action</td>
 </tr>
 </thead>
 <tbody>
 @foreach($blogs as $blog)
 <tr>
 <td>{{$blog->id}}</td>
 <td>{{$blog->title}}</td>
 <td>{{$blog->category}}</td>
 <td>{{$blog->description}}</td>
 <td>Edit</td>
 <td>Delete</td>
 </tr>
 @endforeach
 </tbody>
 </table>
<div>
@endsection

Now all code is ready but we need to add 1 line of code to prevent the blog controller without authentication or without login

just add the following constructor method in BlogController class

 public function __construct()
 {
$this->middleware('auth');
 }

this constructor method will call very first when user will try to access any of BlogController class method, and the middleware will check whether user is logged in then only it will allow to access that method otherwise it will redirect to login page automatically.


After It Run your Code and you will able to create and listing your created blogs/articles. but the Edit and Delete links are not working right now, If you want that also working then please comment here or message me. If we get multiple requests then definitely i will write its part 2 article


Thanks very much for reading this blog, if you have any doubt about it then let me know in comments or by messaging me.

Following is the final code for BlogController.php

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use App\Blog;



class BlogController extends Controller
{
 public function __construct()
 {
 $this->middleware('auth');
 }

 public function createBlog()
{
 return view('blog/create');
}


public function saveBlog(Request $request)
 {
 $blog = new Blog();
 
 $this->validate($request, [
 'title'=>'required',
'category'=>'required',
 'description'=> 'required'
 ]);
 
 $blog->createBlog($request->all());
 return redirect('blog/index')->with('success', 'New blog has been created successfully :)');
 }

 public function showAllBlogs()
 {
 $blogs = Blog::where('user_id', auth()->user()->id)->get();
 
 return view('blog.index',compact('blogs'));
 }

}

Introduction

In a previous article we learned about Creating a PHP Daemon Service. Now we are going to learn how to use methods to perform IPC - Inter-Process Communication - to communicate with daemon processes.

Message Queues

In the world of UNIX, there is an incredible variety of ways to send a message or a command to a daemon script and vice versa. But first I want to talk only about message queues - "System V IPC Messages Queues".

A long time ago I learned that a queue can be either in the System V IPC implementation, or in the POSIX implementation. I want to comment only about the System V implementation, as I know it better.

Lets get started. At the "normal" operating system level, queues are stored in memory. Queue data structures are available to all system programs. Just as in the file system, it is possible to configure queues access rights and message size. Usually a queue message size is small, less than 8 KB.

This introductory part is over. Lets move on to the practice with same example scripts.queue-send.php

$key = ftok(__FILE__, 'A'); 
$queue = msg_get_queue($key);

msg_send($queue, 1, 'message, type 1');
msg_send($queue, 2, 'message, type 2');
msg_send($queue, 3, 'message, type 3');
msg_send($queue, 1, 'message, type 1');

echo "send 4 messages
";

queue-receive.php

$key = ftok('queue-send.php', 'A');
$queue = msg_get_queue($key);

for ($i = 1; $i <= 3; $i++) {
 echo "type: {$i}
";

 while ( msg_receive($queue, $i, $msgtype, 4096, $message, false, MSG_IPC_NOWAIT) ) {
 echo "type: {$i}, msgtype: {$msgtype}, message: {$message}
";
 }
}

Lets run on the first stage of the file queue-send.php, and then queue-receive.php.

u% php queue-send.php
send 4 messages
u% php queue-receive.php
type: 1
type: 1, msgtype: 1, message: s:15:"message, type 1";
type: 1, msgtype: 1, message: s:15:"message, type 1";
type: 2
type: 2, msgtype: 2, message: s:15:"message, type 2";
type: 3
type: 3, msgtype: 3, message: s:15:"message, type 3";

You may notice that the messages have been grouped. The first group gathered 2 messages of the first type, and then the remaining messages.

If we would have indicated to receive messages of type 0, you would get all messages, regardless of the type.

while (msg_receive($queue, $i, $msgtype, 4096, $message, false, MSG_IPC_NOWAIT)) {

Here it is worth noting another feature of the queues: if we do not use the constant MSG_IPC_NOWAIT in the script and run the script queue-receive.php from a terminal, and then run periodically the file queue-send.php, we see how a daemon can effectively use this to wait jobs.queue-receive-wait.php

$key = ftok('queue-send.php', 'A');
$queue = msg_get_queue($key);

while ( msg_receive($queue, 0, $msgtype, 4096, $message) ) {
 echo "msgtype: {$msgtype}, message: {$message}
";
}

Actually that is the most interesting information of all I have said. There are also functions to get statistics, disposal and checking for the existence of queues.

Lets now try to write a daemon listening to a queue:queue-daemon.php

$pid = pcntl_fork();
$key = ftok('queue-send.php', 'A');
$queue = msg_get_queue($key);

if ($pid == -1) {
 exit;
} elseif ($pid) {
 exit;
} else {
 while ( msg_receive($queue, 0, $msgtype, 4096, $message) ) {
 echo "msgtype: {$msgtype}, message: {$message}
";
 }
}

posix_setsid();

Shared Memory

We have learned to work with queues, with which you can send small system messages. But then we may certainly be faced with the task of transmitting large amounts of data. My favorite type of system, System V, has solved the problem of rapid transmission and preservation of large data in memory using a mechanism calledShared Memory.

In short, the data in the Shared Memory lives until the system is rebooted. Since the data is in memory, it works much faster than if it was stored in a database somewhere in a file, or, God forgive me on a network share.

Lets try to write a simple example of data storage.shared-memory-write-base.php

$id = ftok(__FILE__, 'A');


$shmId = shm_attach($id);

$var = 1;

if (shm_has_var($shmId, $var)) {
 $data = (array) shm_get_var($shmId, $var);
} else {
 $data = array();
}

$data[time()] = file_get_contents(__FILE__);

shm_put_var($shmId, $var, $data);

Run this script several times to save the value in memory. Now lets write a script only to read from the memory.shared-memory-read-base.php

$id = ftok(__DIR__ . '/shared-memory-write-base.php', 'A');
$shmId = shm_attach($id);
$var = 1; 

if (shm_has_var($shmId, $var)) {
 $data = (array) shm_get_var($shmId, $var);
} else {
 $data = array();
}

foreach ($data as $key => $value) {
 $path = "/tmp/$key.php";
 file_put_contents($path, $value);

 echo $path . PHP_EOL;
} 

Semaphores

So, in general terms, it should be clear for you by now how to work with shared memory. The only problems left to figure out are about a couple of nuances, such as: "What to do if two processes want to record one block of memory?" Or "How to store binary files of any size?".

To prevent simultaneous accesses we will use semaphores. Semaphores allow us to flag that we want to have exclusive access to some resource, like for instance a shared memory block. While that happens other processes will wait for their turn on semaphore.

In this code it explained clearly:shared-memory-semaphors.php

$id = ftok(__FILE__, 'A');

$semId = sem_get($id);

sem_acquire($semId);

$data = file_get_contents(__DIR__.'/06050396.JPG', FILE_BINARY);

$shmId = shm_attach($id, strlen($data)+4096);
$var = 1;
 
if (shm_has_var($shmId, $var)) {
 $data = shm_get_var($shmId, $var);

 $filename = '/tmp/' . time();
 file_put_contents($filename, $data, FILE_BINARY);

 shm_remove($shmId);
} else {
 shm_put_var($shmId, $var, $data); 
}

sem_release($semId); 

Now you can use the md5sum command line utility to compare two files, the original and the saved file. Or, you can open the file in image editor or whatever prefer to compare the images.

With this we are done with shared memory and semaphores. As your homework I want to ask you to write code that a demon will use semaphores to access shared memory.

Conclusion

Exchanging data between the daemons is very simple. This article described two options for data exchange: message queues and shared memory.

Post a comment here if you have questions or comments about how to exchange data with daemon services in PHP.

What is a Daemon?

The term daemon was coined by the programmers of Project MAC at MIT. It is inspired on Maxwell's demon in charge of sorting molecules in the background. The UNIX systems adopted this terminology for daemon programs.

It also refers to a character from Greek mythology that performs the tasks for which the gods do not want to take. As stated in the "Reference System Administrator UNIX", in ancient Greece, the concept of "personal daemon" was, in part, comparable to the modern concept of "guardian angel." BSD family of operating systems use the image as a demon's logo.

Daemons are usually started at machine boot time. In the technical sense, a demon is considered a process that does not have a controlling terminal, and accordingly there is no user interface. Most often, the ancestor process of the deamon is init - process root on UNIX, although many daemons run from special rcd scripts started from a terminal console.

Richard Stevenson describes the following steps for writing daemons:
    . Resetting the file mode creation mask to 0 function umask(), to mask some bits of access rights from the starting process.
    . Cause fork() and finish the parent process. This is done so that if the process was launched as a group, the shell believes that the group finished at the same time, the child inherits the process group ID of the parent and gets its own process ID. This ensures that it will not become process group leader.
    . Create a new session by calling setsid(). The process becomes a leader of the new session, the leader of a new group of processes and loses the control of the terminal.
    . Make the root directory of the current working directory as the current directory will be mounted.
    . Close all file descriptors.
    . Make redirect descriptors 0,1 and 2 (STDIN, STDOUT and STDERR) to /dev/null or files /var/log/project_name.out because some standard library functions use these descriptors.
    . Record the pid (process ID number) in the pid-file: /var/run/projectname.pid.
    . Correctly process the signals and SigTerm SigHup: end with the destruction of all child processes and pid - files and / or re-configuration.

How to Create Daemons in PHP

To create demons in PHP you need to use the extensions pcntl and posix. To implement the fast communication withing daemon scripts it is recommended to use the extension libevent for asynchronous I/O.

Lets take a closer look at the code to start a daemon:

umask(0);
$pid = pcntl_fork(); 
if ($pid < 0) {
 print('fork failed');
 exit 1; 
}

After a fork, the execution of the program works as if there are two branches of the code, one for the parent process and the second for the child process. What distinguishes these two processes is the result value returned the fork() function call. The parent process ID receives the newly created process number and the child process receives a 0.

if ($pid > 0) { echo "daemon process started
";
 exit;
} 
 
$sid = posix_setsid();
if ($sid < 0) {
 exit 2;
}

chdir('/');
file_put_contents($pidFilename, getmypid() ); 
run_process();

The implementation of step 5 "to close all file descriptors" can be done in two ways. Well, closing all file descriptors is difficult to implement in PHP. You just need to open any file descriptors before fork(). Second, you can override the standard output to an error log file using init_set() or use buffering using ob_start() to a variable and store it in log file:

ob_start();
var_dump($some_object);
$content = ob_get_clean();
fwrite($fd_log, $content); 

Typically, ob_start() is the start of the daemon life cycle and ob_get_clean() and fwrite() calls are the end. However, you can directly override STDIN, STDOUT and STDERR:

ini_set('error_log', $logDir.'/error.log');
fclose(STDIN); 
fclose(STDOUT);
fclose(STDERR);
$STDIN = fopen('/dev/null', 'r'); 
$STDOUT = fopen($logDir.'/application.log', 'ab');
$STDERR = fopen($logDir.'/application.error.log', 'ab');

Now, our process is disconnected from the terminal and the standard output is redirected to a log file.

Handling Signals

Signal processing is carried out with the handlers that you can use either via the library pcntl (pcntl_signal_dispatch()), or by using libevent. In the first case, you must define a signal handler:

function sig_handler($signo)
{
 global $fd_log;
 switch ($signo) {
 case SIGTERM:
 fclose($fd_log); unlink($pidfile); exit;
 break;
 case SIGHUP:
 init_data(); break;
 default:
 }
}


pcntl_signal(SIGTERM, "sig_handler");
pcntl_signal(SIGHUP, "sig_handler");

Note that signals are only processed when the process is in an active mode. Signals received when the process is waiting for input or in sleep mode will not be processed. Use the wait function pcntl_signal_dispatch(). We can ignore the signal using flag SIG_IGN: pcntl_signal(SIGHUP, SIG_IGN); Or, if necessary, restore the signal handler using the flag SIG_DFL, which was previously installed by default: pcntl_signal(SIGHUP, SIG_DFL);

Asynchronous I/O with Libevent

In the case you use blocking input / output signal processing is not applied. It is recommended to use the library libevent which provides non-blocking as input / output, processing signals, and timers. Libevent library provides a simple mechanism to start the callback functions for events on file descriptor: Write, Read, Timeout, Signal.

Initially, you have to declare one or more events with an handler (callback function) and attach them to the basic context of the events:

$base = event_base_new();
$event = event_new();
$errno = 0;
$errstr = '';
$socket = stream_socket_server("tcp://$IP:$port", $errno, $errstr);
stream_set_blocking($socket, 0);
event_set($event, $socket, EV_READ | EV_PERSIST, 'onAccept', $base);

Function handlers 'onRead', 'onWrite', 'onError' must implement the processing logic. Data is written into the buffer, which is obtained in the non-blocking mode:

function onRead($buffer, $id)
{
 while($read = event_buffer_read($buffer, 256)) {
 var_dump($read);
 }
}

The main event loop runs with the function event_base_loop($base);. With a few lines of code, you can exit the handler only by calling: event_base_loobreak(); or after the specified time (timeout) event_loop_exit();.

Error handling deals with failure Events:

function onError($buffer, $error, $id)
{
 global $id, $buffers, $ctx_connections;
 event_buffer_disable($buffers[$id], EV_READ | EV_WRITE);
 event_buffer_free($buffers[$id]);
 fclose($ctx_connections[$id]);
 unset($buffers[$id], $ctx_connections[$id]);
}

It should be noted the following subtlety: Working with timers is only possible through the file descriptor. The example of official the documentation does not work. Here is an example of processing that runs at regular intervals.

$event2 = event_new();
$tmpfile = tmpfile();
event_set($event2, $tmpfile, 0, 'onTimer', $interval);
$res = event_base_set($event2, $base);
event_add($event2, 1000000 * $interval);

With this code we can have a working timer finishes only once. If we need a "permanent" Timer, using the function onTimer we need create a new event each time, and reassign it to process through a "period of time":

function onTimer($tmpfile, $flag, $interval)
{
 $global $base, $event2;

 if ($event2) {
 event_delete($event2);
 event_free($event2);
 }

 call_user_function(‘process_data’,$args);

 $event2 = event_new();
 event_set($event2, $tmpfile, 0, 'onTimer', $interval);
 $res = event_base_set($event2, $base);
 event_add($event2, 1000000 * $interval);
}

At the end of the daemon we must release all previously allocated resources:

event_delete($event);
event_free($event);
event_base_free($base);

event_base_set($event, $base);
event_add($event);

Also it should be noted that for the signal processing handler is set the flag EV_SIGNAL: event_set($event, SIGHUP, EV_SIGNAL, 'onSignal', $base);

If needed constant signal processing, it is necessary to set a flag EV_PERSIST. Here follows a handler for the event onAccept, which occurs when a new connection is a accepted on a file descriptor:

function onAccept($socket, $flag, $base) {
 global $id, $buffers, $ctx_connections;
 $id++;
 $connection = stream_socket_accept($socket);
 stream_set_blocking($connection, 0);
 $buffer = event_buffer_new($connection, 'onRead', NULL, 'onError', $id);
 event_buffer_base_set($buffer, $base);
 event_buffer_timeout_set($buffer, 30, 30);
 event_buffer_watermark_set($buffer, EV_READ, 0, 0xffffff); event_buffer_priority_set($buffer, 10); event_buffer_enable($buffer, EV_READ | EV_PERSIST); $ctx_connections[$id] = $connection;
 $buffers[$id] = $buffer;
}

Monitoring a Daemon

It is good practice to develop the application so that it was possible to monitor the daemon process. Key indicators for monitoring are the number of items processed / requests in the time interval, the speed of processing with queries, the average time to process a single request or downtime.

With the help of these metrics can be understood workload of our demon, and if it does not cope with the load it gets, you can run another process in parallel, or for running multiple child processes.

To determine these variables need to check these features at regular intervals, such as once per second. For example downtime is calculated as the difference between the measurement interval and total time daemon.

Typically downtime is determined as a percentage of a measurement interval. For example, if in one second were executed 10 cycles with a total processing time of 50ms, the time will be 950ms or 95%.

Query performance wile be 10rps (request per second). Average processing time of one request: the ratio of the total time spent on processing requests to the number of requests processed, will be 5ms.

These characteristics, as well as additional features such as memory stack size queue, number of transactions, the average time to access the database, and so on.

An external monitor can be obtain data through a TCP connection or unix socket, usually in the format of Nagios or zabbix, depending on the monitoring system. To do this, the demon should use an additional system port.

As mentioned above, if one worker process can not handle the load, usually we run in parallel multiple processes. Starting a parallel process should be done by the parent master process that uses fork() to launch a series of child processes.

Why not run processes using exec() or system()? Because, as a rule, you must have direct control over the master and child processes. In this case, we can handle it via interaction signals. If you use the exec command or system, then launch the initial interpreter, and it has already started processes that are not direct descendants of the parent process.

Also, there is a misconception that you can make a demon process through command nohup. Yes, it is possible to issue a command: nohup php mydaemon.php -master >> /var/log/daemon.log 2 >> /var/log/daemon.error.log &

But, in this case, would be difficult to perform log rotation, as nohup "captures" file descriptors for STDOUT / STDERR and release them only at the end of the command, which may overload of the process or the entire server. Overload demon process may affect the integrity of data processing and possibly cause partial loss of some data.

Starting a Daemon

Starting the daemon must happen either automatically at boot time, or with the help of a "boot script."

All startup scripts are usually in the directory /etc/rc.d. The startup script in the directory service is made /etc/init.d/ . Run command start service myapp or start group /etc/init.d/myapp depending on the type of OS.

Here is a sample script text:

#! /bin/sh
#
$appdir = /usr/share/myapp/app.php
$parms = --master –proc=8 --daemon
export $appdir
export $parms
if [ ! -x appdir ]; then
 exit 1
fi

if [ -x /etc/rc.d/init.d/functions ]; then
 . /etc/rc.d/init.d/functions
fi

RETVAL=0

start () {
 echo "Starting app"
 daemon /usr/bin/php $appdir $parms 
 RETVAL=$?
 [ $RETVAL -eq 0 ] && touch /var/lock/subsys/mydaemon
 echo
 return $RETVAL
}

stop () {
 echo -n "Stopping $prog: "
 killproc /usr/bin/fetchmail
 RETVAL=$?
 [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/mydaemon
 echo
 return $RETVAL
}

case in
 start)
 start
 ;;
 stop)
 stop
 ;;
 restart)
 stop
 start
 ;;
 status)
 status /usr/bin/mydaemon
 ;;
 *)
 echo "Usage: 

php
if (is_file('app.phar')) {
unlink('app.phar');
}
$phar = new Phar('app.phar', 0, 'app.phar');
$phar->compressFiles(Phar::GZ);
$phar->setSignatureAlgorithm (Phar::SHA1);
$files = array();
$files['bootstrap.php'] = './bootstrap.php';
$rd = new RecursiveIteratorIterator(new RecursiveDirectoryIterator('.'));
foreach($rd as $file){
if ($file->getFilename() != '..' && $file->getFilename() != '.' && $file->getFilename() != __FILE__) {
if ( $file->getPath() != './log'&& $file->getPath() != './script'&& $file->getPath() != '.')
$files[substr($file->getPath().DIRECTORY_SEPARATOR.$file->getFilename(),2)] =
$file->getPath().DIRECTORY_SEPARATOR.$file->getFilename();
}
}
if (isset($opt['version'])) {
$version = $opt['version'];
$file = "buildFromIterator(new ArrayIterator($files));
$phar->setStub($phar->createDefaultStub('bootstrap.php'));
$phar = null;
}

 {start|stop|restart|status}"
 ;;
 
RETVAL=$?
exit $RETVAL

Distributing Your PHP Daemon

To distribute a daemon it is better to pack it in a single phar archive module. The assembled module should include all the necessary PHP and .ini files.

Below is a sample build script:

#php app.phar
myDaemon version 0.1 Debug
usage:
--daemon – run as daemon
--debug – run in debug mode
--settings – print settings
--nofork – not run child processes
--check – check dependency modules
--master – run as master
--proc=[8] – run child processes

Additionally, it may be advisable to make a PEAR package as a standard unix-console utility that when run with no arguments prints its own usage instruction:
[NMD%%CODE%%]

Conclusion

Creating daemons in PHP it is not hard but to make them run correctly it is important to follow the steps described in this article.

Post a comment here if you have questions or comments on how to create daemon services in PHP.

For those of you not familiar with the concept of underclocking: it's the opposite of overclocking, that is, you don't speed up CPU but instead slow it down..

What for?

Ask the underclockers, I'm totally not sure. Actually, hanging around the Web these days leaves a feeling that nearly every website out there must have been underclocked, but most of the time it's about tons of unnecessary images, megabytes of javascript (of which hardly a hundred kilobytes gets actually executed), and all that. In this post I will, however, tell you about a server-side approach to underclocking, with a help of our good old friend - the MySQL Database Server.

Today I had a nice chat in my client's development telegram channel. The two other devs, R** and V**, were making a switch of the old image API app to a new MySQL server. A couple of days before that, we have discussed a plan, it was as dumb as possible, just as I like it. Super-simple clear steps that a five-year-old can make. Switch to readonly mode (stop uploads), dump DB, restore the dump on the new server, update database connection details, turn off readonly mode. What could possibly go wrong?

Nothing. Except that it did go wrong. The app that I'm talking about, is a really ancient piece of what is gently called "legacy". Once the app was back to normal again, we noticed a significant slowdown on every page that made use of images. Before that point, I never got to that app and/or its database. I logged in to the MySQL console, and started investigating, at the same point chatting with colleagues.

Me: Is that really important that the tables are MyISAM? It's 2018, you know.. There are dozens of queries in queue waiting for table-level locks.

R**: Are they MyISAM? Really?

Me: Yes.. Any objection against converting them to InnoDB? With the current state of the website, with all those tons of Gateway Timeouts, it's not going to make it worse if I do it right now..

    . minutes later:

Me: Nah, it didn't help a lot.. But, looking at the SHOW PROCESSLIST output, I see something weird. What, do you think, this query does? SELECT LAST_INSERT_ID() FROM images? 

R**: ehh... Gets you the last AUTO_INCREMENT id from images table?

Me: Let's play another good news bad news joke.. Good news: you're right, it gets you the last AUTO_INCREMENT id. Bad news: it's not for table, it's for the session. Worse news: this query gets you the last AUTO_INCREMENT id and does it exactly as many times as there are rows in the images table. how many are there?

R**: about 8mln. #@%&! It's sending 8mln rows on every image upload, through the network!

Me: Bingo! 8mln rows, with one and the same integer value in all of them.

R**: Ouch... Aaaand... Before today, it was not an issue. Because the database was on the same server as the application..

Me: Exactly, it used the loopback interface, and now it's using ehternet, which, apparently, doesn't have a super good bandwidth. We don't have a gigabit channel between servers, do we?

R**: No, it's 100 Mbit

Me: Are you fixing the query, BTW?

R**: yeah, man, deploying it...

Another 10 minutes later, problem is gone, performance is back to normal.

What conclusions can one make from this story?

I can think of two at least:
First: never underestimate legacy code. The ways it can move to bite you in the ass, are mysterious.
Second: if you're working with MySQL or another RDBMS, learn SQL, learn the specific SQL dialect you're using and learn how to trouble shoot issues. In this case, I did not need to look in the PHP code at all in order to help my fellow colleagues out. You can also generalize this principle as "you have to know the tools you're using".

Happy optimizing, folks! Comments appreciated!