PHP & Web Development Blogs

For those of you not familiar with the concept of underclocking: it's the opposite of overclocking, that is, you don't speed up CPU but instead slow it down..

What for?

Ask the underclockers, I'm totally not sure. Actually, hanging around the Web these days leaves a feeling that nearly every website out there must have been underclocked, but most of the time it's about tons of unnecessary images, megabytes of javascript (of which hardly a hundred kilobytes gets actually executed), and all that. In this post I will, however, tell you about a server-side approach to underclocking, with a help of our good old friend - the MySQL Database Server.

Today I had a nice chat in my client's development telegram channel. The two other devs, R** and V**, were making a switch of the old image API app to a new MySQL server. A couple of days before that, we have discussed a plan, it was as dumb as possible, just as I like it. Super-simple clear steps that a five-year-old can make. Switch to readonly mode (stop uploads), dump DB, restore the dump on the new server, update database connection details, turn off readonly mode. What could possibly go wrong?

Nothing. Except that it did go wrong. The app that I'm talking about, is a really ancient piece of what is gently called "legacy". Once the app was back to normal again, we noticed a significant slowdown on every page that made use of images. Before that point, I never got to that app and/or its database. I logged in to the MySQL console, and started investigating, at the same point chatting with colleagues.

Me: Is that really important that the tables are MyISAM? It's 2018, you know.. There are dozens of queries in queue waiting for table-level locks.

R**: Are they MyISAM? Really?

Me: Yes.. Any objection against converting them to InnoDB? With the current state of the website, with all those tons of Gateway Timeouts, it's not going to make it worse if I do it right now..

    . minutes later:

Me: Nah, it didn't help a lot.. But, looking at the SHOW PROCESSLIST output, I see something weird. What, do you think, this query does? SELECT LAST_INSERT_ID() FROM images? 

R**: ehh... Gets you the last AUTO_INCREMENT id from images table?

Me: Let's play another good news bad news joke.. Good news: you're right, it gets you the last AUTO_INCREMENT id. Bad news: it's not for table, it's for the session. Worse news: this query gets you the last AUTO_INCREMENT id and does it exactly as many times as there are rows in the images table. how many are there?

R**: about 8mln. #@%&! It's sending 8mln rows on every image upload, through the network!

Me: Bingo! 8mln rows, with one and the same integer value in all of them.

R**: Ouch... Aaaand... Before today, it was not an issue. Because the database was on the same server as the application..

Me: Exactly, it used the loopback interface, and now it's using ehternet, which, apparently, doesn't have a super good bandwidth. We don't have a gigabit channel between servers, do we?

R**: No, it's 100 Mbit

Me: Are you fixing the query, BTW?

R**: yeah, man, deploying it...

Another 10 minutes later, problem is gone, performance is back to normal.

What conclusions can one make from this story?

I can think of two at least:
First: never underestimate legacy code. The ways it can move to bite you in the ass, are mysterious.
Second: if you're working with MySQL or another RDBMS, learn SQL, learn the specific SQL dialect you're using and learn how to trouble shoot issues. In this case, I did not need to look in the PHP code at all in order to help my fellow colleagues out. You can also generalize this principle as "you have to know the tools you're using".

Happy optimizing, folks! Comments appreciated!

To say that we have been hard at work here at Nomad PHP, or that I'm excited about these three announcements would be a tremendous understatement. Over the past several months, behind the scenes, we've been working to bring even more features and benefits to Nomad PHP - these have already included unlimited streaming of all past meetings and access to PHP Architect.

Available today, however, you'll also have access to online, live workshops - as well as soon have the ability to stream select PHP conferences live, and finally to prove the knowledge you have gained through our online certification.

Online, Live Workshops

Like our online meetings, we are excited to announce that available today you can participate in online, live, and interactive workshops. Our first workshop will feature Michael Stowe, author of Undisturbed REST: a guide to Designing the Perfect API as he demonstrates how to build the perfect API using modern technologies and techniques.

Additional workshops will be announced as we continue, with a minimum of one workshop per quarter. These workshops will be part of your Nomad PHP subscription, and will be recorded for later viewing.

Nomad PHP Certification

With the many changes impacting the PHP ecosystem, we're proud to announce the ability to prove your knowledge with our online certification. Each certification is made up numerous, randomly selected questions to be completed within a specific time frame. Depending on the exam it may or may not be proctored, but all exams monitor user activity to ensure compliance.

To pass the exam, a passing grade (specified on each exam) must be completed for each section within the allotted time frame. Failure to complete or pass any section will result in a failing grade for the entire exam.

Upon completion, you will receive a digital certification with verification to post on LinkedIn or your website, as well as having your Nomad PHP updated to show the passed certification.

Initial certification exams will include PHP Developer Level I, PHP Engineer Level II, and API Specialist Level I. The PHP Developer exam will cover core components of PHP, the Engineer will cover a broad spectrum of topics including modern technologies, and the API Specialist will cover REST design and architecture practices.

All three exams will be available by January 31, 2019, and will be included with a Nomad PHP subscription.

Stream Select PHP Conferences Live

One of the primary goals of Nomad PHP is to bring the community together, and allow users all over the country to participate in conference level talks. What better way to do this than to bring community conferences online?

Like our traditional talks, these conferences and select conference sessions will be live-streamed as part of your Nomad PHP subscription, allowing you to participate in real-time with in-person conference attendees.

The first conference to be streamed will be DayCamp4Developers: Beyond Performance on January 18, 2018. Additional conferences to be streamed will be announced shortly.

Community and Corporate Sponsorships

With these new additions to Nomad PHP, now is the perfect time to take advantage of our new Community and Corporate sponsorships.

Your support of Nomad PHP not only makes all the above possible, but allows Nomad PHP to continue to serve and give back the community. We're proud, that despite operating at a loss, to have already contributed over$4,000 to the PHP community in the last 5 months.

To learn more about the sponsorship and community opportunities we have available, please visit our Advertising section.

Other Ways to Support Nomad PHP

Of course, while financial support helps us keep afloat and do more for the community, there are even more, and just as important ways to support Nomad PHP. Please consider linking to Nomad PHP, or sharing the service with your friends.

As a developer, you've probably at least heard something about CI - Continuous integration. And if you haven't - you better fix it ASAP, because that's something awesome to have on your skill list and can get extremely helpful in your everyday work. This post will focus on CI for PHP devs, and specifically, on CI implementation from Gitlab. I will suppose you know the basics of Git, PHP, PHPUnit, Docker and unix shell. Intended audience - intermediate PHP devs.
Adding something to your workflow must serve a purpose. In this case the goal is to automate routine tasks and achieve better quality control. Even a basic PHP project IMO needs the following:
* linter) checks (cannot merge changes that are invalid on the syntax level)
* Code style checks
* Unit and integration tests
All of those can be just run eventually, of course. But I prefer an automated CI approach even in my personal projects because it leads to a higher level of discipline, you simply can't avoid following a set of rules that you've developed. Also, it reduces a risk of releasing a bug or regression, thus improving quality.
Gitlab is as generous as giving you their CI for free, even for your private repos. At this point it is starting to look as advertising, therefore a quick comparison table for Gitlab, Github, Bitbucket. AFAIK, Github does not have a built-in solution, instead it is easily integrated with third parties, of which Travis CI seems to be the most popular - I will therefore mention Travis here.

Public repositories (OSS projects). All 3 providers have a free offer for the open-source community!

| Provider | Limits |
|---|---|
| Gitlab | 2,000 CI pipeline minutes per group per month, shared runners |
| Travis | Apparently unlimited |
| Bitbucket| 50 min/month, max 5 users, File storage <= 1Gb/month |

Private repositories

| Provider | Price | Limits |
|---|---|---|
| Gitlab | Free | 2,000 CI pipeline minutes per group per month, shared runners |
| Travis | $69/month | Unlimited builds, 1 job at a time |
| Bitbucket| Free | 50 min/month, max 5 users, File storage <= 1Gb/month |

Getting started

I made a small project based on Laravel framework and called it "ci-showcase". I work in Linux environment, and the commands I use in the examples, are for linux shell. They should be pretty much the same on Mac and nearly the same on Windows though.

composer create-project laravel/laravel ci-showcase

Next, I went to gitlab website and created a new public project: https://gitlab.com/crocodile2u/ci-showcase. Cloned the repo and copied all files and folders from the newly created project - the the new git repo. In the root folder, I placed a .gitignore file:

.idea
vendor
.env

Then the .env file:

APP_ENV=development

Then I generated the application encryption key: php artisan key:generate, and then I wanted to verify that the primary setup works as expected: ./vendor/bin/phpunit, which produced the output OK (2 tests, 2 assertions). Nice, time to commit this: git commit && git push

At this point, we don't yet have any CI, let's do something about it!

Adding .gitlab-ci.yml

Everyone going to implement CI with Gitlab, is strongly encouraged to bookmark this page: https://docs.gitlab.com/ee/ci/README.html. I will simply provide a short introduction course here plus a bit of boilerplate code to get you started easier.
First QA check that we're going to add is PHP syntax check. PHP has a built-in linter, which you can invoke like this: php -l my-file.php. This is what we're going to use. Because the php -l command doesn't support multiple files as arguments, I've written a small wrapper shell script and saved it to ci/linter.sh:

#!/bin/sh
files=<code>sh ci/get-changed-php-files.sh | xargs</code>last_status=0
status=0
# Loop through changed PHP files and run php -l on each
for f in "$files" ; do message=<code>php -l $f</code> last_status="$?" if [ "$last_status" -ne "0" ]; then # Anything fails -> the whole thing fails echo "PHP Linter is not happy about $f: $message" status="$last_status" fi
done
if [ "$status" -ne "0" ]; then echo "PHP syntax validation failed!"
fi
exit $status

Most of the time, you don't actually want to check each and every PHP file that you have. Instead, it's better to check only those files that have been changed. The Gitlab pipeline runs on every push to the repository, and there is a way to know which PHP files have been changed. Here's a simple script, meet ci/get-changed-php-files.sh:

#!/bin/sh
# What's happening here?
#
# 1. We get names and statuses of files that differ in current branch from their state in origin/master.
# These come in form (multiline)
# 2. The output from git diff is filtered by unix grep utility, we only need files with names ending in .php
# 3. One more filter: filter *out* (grep -v) all lines starting with R or D.
# D means "deleted", R means "renamed"
# 4. The filtered status-name list is passed on to awk command, which is instructed to take only the 2nd part
# of every line, thus just the filename
git diff --name-status origin/master | grep '\.php$' | grep -v "^[RD]" | awk '{ print }'

These scripts can easily be tested in your local environment ( at least if you have a Linux machine, that is ;-) ).
Now, as we have our first check, we'll finally create our .gitlab-ci.yml. This is where your pipeline is declared using YAML notation:

# we're using this beautiful tool for our pipeline: https://github.com/jakzal/phpqa
image: jakzal/phpqa:alpine
# For this sample pipeline, we'll only have 1 stage, in real-world you would like to also add at least "deploy"
stages: - QA
linter:
stage: QA
# this is the main part: what is actually executed
script:	- sh ci/get-changed-php-files.sh | xargs sh ci/linter.sh

The first line is image: jakzal/phpqa:alpine and it's telling Gitlab that we want to run our pipeline using a PHP-QA utility by jakzal. It is a docker image containing PHP and a huge variety of QA-tools. We declare one stage - QA, and this stage by now has just a single job named linter. Every job can have it's own docker image, but we don't need that for the purpose of this tutorial. Our project reaches Step 2. Once I had pushed these changes, I immediately went to the project's CI/CD page. Aaaand.... the pipeline was already running! I clicked on the linter job and saw the following happy green output:

Running with gitlab-runner 11.9.0-rc2 (227934c0) on docker-auto-scale ed2dce3a
Using Docker executor with image jakzal/phpqa:alpine ...
Pulling docker image jakzal/phpqa:alpine ...
Using docker image sha256:12bab06185e59387a4bf9f6054e0de9e0d5394ef6400718332c272be8956218f for jakzal/phpqa:alpine ...
Running on runner-ed2dce3a-project-11318734-concurrent-0 via runner-ed2dce3a-srm-1552606379-07370f92...
Initialized empty Git repository in /builds/crocodile2u/ci-showcase/.git/
Fetching changes...
Created fresh repository.
From https://gitlab.com/crocodile2u/ci-showcase * [new branch] master -> origin/master * [new branch] step-1 -> origin/step-1 * [new branch] step-2 -> origin/step-2
Checking out 1651a4e3 as step-2...
Skipping Git submodules setup
$ sh ci/get-changed-php-files.sh | xargs sh ci/linter.sh
Job succeeded

It means that our pipeline was successfully created and run!

PHP Code Sniffer.

PHP Code Sniffer is a tool for keeping app of your PHP files in one uniform code style. It has a hell of customizations and settings, but here we will only perform simple check for compatibilty with PSR-2 standard. A good practice is to create a configuration XML file in your project. I will put it in the root folder. Code sniffer can use a few file names, of which I prefer phpcs.xml:

<?xml version="1.0"?>
/resources

I also will append another section to .gitlab-ci.yml:

code-style:	stage: QA	script:	# Variable $files will contain the list of PHP files that have changes	- files=<code>sh ci/get-changed-php-files.sh</code> # If this list is not empty, we execute the phpcs command on all of them - if [ ! -z "$files" ]; then echo $files | xargs phpcs; fi

Again, we check only those PHP files that differ from master branch, and pass their names to phpcs utility. That's it, Step 3 is finished! If you go to see the pipeline now, you will notice that linter and code-style jobs run in parallel.

Adding PHPUnit

Unit and integration tests are essential for a successful and maintaiable modern software project. In PHP world, PHPUnit is de facto standard for these purposes. The PHPQA docker image already has PHPUnit, but that's not enough. Our project is based on Laravel, which means it depends on a bunch of third-party libraries, Laravel itself being one of them. Those are installed into vendor folder with composer. You might have noticed that our .gitignore file has vendor folder as one of it entries, which means that it is not managed by the Version Control System. Some prefer their dependencies to be part of their Git repository, I prefer to have only the composer.json declarations in Git. Makes the repo much much smaller than the other way round, also makes it easy to avoid bloating your production builds with libraries only needed for development.
Composer is also included into PHPQA docker image, and we can enrich our .gitlab-ci.yml:

test:	stage: QA	cache:	key: dependencies-including-dev	paths: - vendor/	script:	- composer install	- ./vendor/bin/phpunit

PHPUnit requires some configuration, but in the very beginning we used composer create-project to create our project boilerplate.laravel/laravel package has a lot of things included in it, and phpunit.xml is also one of them. All I had to do was to add another line to it:

xml

APP_KEY enironment variable is essential for Laravel to run, so I generated a key with php artisan key:generate.
git commit & git push, and we have all three jobs on theQA stage!

Checking that our checks work

In this branch I intentionally added changes that should fail all three job in our pipeline, take a look at git diff. And we have this out from the pipeline stages:Linter:

$ ci/linter.sh
PHP Linter is not happy about app/User.php:
Parse error: syntax error, unexpected 'syntax' (T_STRING), expecting function (T_FUNCTION) or const (T_CONST) in app/User.php on line 11
Errors parsing app/User.php
PHP syntax validation failed!
ERROR: Job failed: exit code 255

**Code-style**:

$ if [ ! -z "$files" ]; then echo $files | xargs phpcs; fi
FILE: ...ilds/crocodile2u/ci-showcase/app/Http/Controllers/Controller.php
----------------------------------------------------------------------
FOUND 0 ERRORS AND 1 WARNING AFFECTING 1 LINE
---------------------------------------------------------------------- 13 | WARNING | Line exceeds 120 characters; contains 129 characters
----------------------------------------------------------------------
Time: 39ms; Memory: 6MB
ERROR: Job failed: exit code 123

**test**:

$ ./vendor/bin/phpunit
PHPUnit 7.5.6 by Sebastian Bergmann and contributors.
F. 2 / 2 (100%)
Time: 102 ms, Memory: 14.00 MB
There was 1 failure:
1) Tests\Unit\ExampleTest::testBasicTest
This test is now failing
Failed asserting that false is true.
/builds/crocodile2u/ci-showcase/tests/Unit/ExampleTest.php:17
FAILURES!
Tests: 2, Assertions: 2, Failures: 1.
ERROR: Job failed: exit code 1

Congratulations, our pipeline is running, and we now have much less chance of messing up the result of our work.

Conclusion

Now you know how to set up a basic QA pipeline for your PHP project. There's still a lot to learn. Pipeline is a powerful tool. For instance, it can make deployments to different environments for you. Or it can build docker images, store artifacts and more! Sounds cool? Then spend 5 minutes of your time and leave a comment, you can also tell me if there is a pipeline topic you would like to be covered in next posts.

Interested in sponsoring? Check out the prospectus

A little history

Several years ago I had the distinct privilege of founding Midwest PHP with Jonathan Sundquist. The goal was simple, to bring an affordable PHP conference to Minnesota and the midwest region.

Midwest PHP was created for one simple reason - there weren't a lot of alternatives, especially affordable ones. At the time, your choices were ZendCon in Silicon Valley, php[tek] in Chicago, or Northeast PHP in Boston. While Northeast PHP formed the blueprint of a community conference - it still required a flight and a costly hotel in Boston. I wanted something where local attendees, college students, and those just beginning in their PHP careers could go to learn, network, and become part of the PHP community.

Shortly after Midwest PHP was formed (originally we were using the name PHPFreeze - until Sundquist told me what a horrible idea it was), Adam Culp launched Sunshine PHP which has become one of the top community focused PHP conferences (but still requires that flight and hotel in Miami). Sundquist and I knew that any reasonable developer would still prefer to attend a conference in a blizzard than enjoy the beautiful Floridian weather (ok, that might not be it, but we still understood the need that existed).

After moving to California for my new job, Jonathan Sundquist continued to run Midwest PHP as more community conferences appeared. With his efforts, and the torch being passed to Mike Willbanks, Midwest PHP celebrated it's seventh consecutive year, becoming the longest continuously running PHP conference (if you go by formed date, if you go by actual conference date Sunshine PHP beats us out by a month).

A renewed focus

Because of the incredible work Jonathan and Mike have done, Midwest PHP has stood the test of time - and the peaks and valleys that come with any conference. With the shifts in the PHP community and the sad loss of several community conferences - we realized the need for Midwest PHP is more now than ever, and to meet that need we needed to reimagine the way the conference operated.

We also realized that the best way to make Midwest PHP accessible was to combine forces, creating a seamless partnership between Nomad PHP and Midwest PHP. Through this partnership we're not only able to stream the event to make it more accessible ($19.95/mo), but also expand the conference.

This year, taking place onApril 2-4, 2020 - Midwest PHP will bring together over 800 developers both in-person and virtually! Making this year truly unique, however, and staying with our purpose of helping new developers be part of the PHP community is abrand new, FREE, beginner track. I'm excited to say we will be giving away 200 tickets to those wishing to attend our Beginner or Learn PHP track!!!

We will also work to keep prices as low as possible as we offer our standard PHP tracks (Everyday PHP and PHP Performance & Security) starting at $250/ person, anda brand new enterprise track geared at developers facing challenges at unprecedented scale starting at $450/ person.

Last but not least, it is our goal with the help of our sponsors to include the workshop day as part of your ticket price - allowing you to get one day of in-depth training, and two more full days of sessions. On top of this, we're also excited to make the Nomad PHP and Nomad JS video libraries available for Standard and Enterprise attendees, providing over 220 additional virtual sessions on demand!

For sponsors

Sponsoring a conference is hard. We understand the challenge of gauging ROI, planning travel, and coordinating outreach. With the combined forces of Midwest PHP and Nomad PHP, we're able to offer sponsors unique plans that maximize their investment - while ensuring the funds go back into the event to create an amazing experience for our attendees.

Beyond Midwest PHP's goal to be the largest PHP conference this year - the included Nomad PHP advertising will help you reach a much larger and broader audience, allowing for follow up advertisements and consistent engagement with the PHP community.

Interested in sponsoring? Check out the prospectus

Next steps

For more information, please visit the Midwest PHP website. The venue, call for papers, and additional information will all be posted there soon.

At Nomad PHP our goal is to empower developers in building a habit of continuous learning - and that means we have a habit of continuous improvement ourselves. Here are just some of the things we've done this year (with much more coming down the road)!

Website Redesign

We've refreshed the look and feel of Nomad PHP to better emphasize the goal of Nomad PHP - to help developers build a habit of continuous learning and grow their careers. This includes numerous usability enhancements as well as a focus on our new book library, blogs, and certification in addition to virtual meetups, workshops, conferences, and on-demand videos.

Free Meetups

As technology has advanced, more and more meetups and usergroups are able to stream their local usergroup meetings.

As our goal has always been to make technology accessible, we are proud to provide free streaming technology for local user groups, and share local user group meetings on our live virtual meetup schedule.

Student and Professional subscribers will continue to have access to our monthly conference level Pro Talks, hands on virtual workshops, and live conference streams in addition to streams by local user groups.

You can find a list of all upcoming talks (free and Pro) on our Live Meetings Page, or add your user group stream here.

Free Subscriber Tier

As our mission has evolved from being the meetup for developers without a meetup group to building an inclusive community of PHP developers where you can network, grow your skills, and share your knowledge with others - we are excited to announce our new Free Tier.

With a free Nomad PHP account you can:

* Stream free meetups

* Watch ad-supported videos in SD

* Read PHP blogs and write your own

* Network with other PHP developers

Create your free developer account to get started.

New Student Tier

To provide the best value, we've also restructured our plans to provide professional online meetings, workshops, and conference streaming to our Student Tier. This will allow students and new developers the chance to learn from the best speakers and top practioners and obtain entry level certifications at the best price possible.

However, with the addition of PHP Books and Magazines, and in order to provide the best value while keeping the Student plan affordable, new Student subscribers will not have access to the PHP Book and Magazine Library, or advanced certifications. These will now require a professional plan.

Student plans start at $12.95/mo

PHP Books and Magazines

We're excited to announce that we have expanded our PHP library. In addition to the ability to read the latest issues of php[architect] magazine, Professional subscribers now have access to read PHP and web development books online.

We're excited to announce the availability of Chris Hartjes' bookThe Grumpy Programmer's Guide to Testing PHP Applications, as well as several titles from Notes for Professionals, andUndisturbed REST: a Guide to Designing the Perfect API.

More titles including exclusive titles will be made available for online reading soon.

You can view our entire PHP Library here.

Blog Updates

We've received a lot of feedback on the blog writing process, and have upgraded several aspects of our blogging software. This includes the ability to save drafts prior to publishing, and the ability to upload, edit, and crop images and videos. We've also added some bug fixes for editing and writing code.

We're also excited to share that members with Student and Professional plans can now have their ownVLOG (video blog) with the ability to screencast/ record video from your webcam within the blog.

To see the most recent blog posts, or write your own, visit the Nomad PHP Blogs.

Certification Updates

We've updated our certifications for better usability and readability. We've also reworked some of the code samples and questions in our Level 1 PHP Certification exam.

You can find our available exams, test your skills, and obtain your Nomad PHP certification here.

Team Management

Our new team manager allows you to easily add or remove team members with your Nomad PHP team subscription. You'll also find real time metrics on how your team is using Nomad PHP, who on your team is investing in their growth and streaming meetups, watching videos, reading books, and earning certifications, and the overall content value consumed by your team.

The Team Manager is available to new teams, and will be made available to existing team managers over the next several weeks.

2020 Roadmap

There's still plenty of more great things coming in 2020. Here are the items at the top of our list:

* Mobile app for offline viewing

* Desktop app for offline viewing

* Nomad PHP member only books

* PHP Level 2 Certification

* Interactive tutorials

* Better video support in blogs

* Ability to schedule blog posts

* Meeting software for local usergroups

* Improved plan management for subscribers
Of course, what's most important to us is what's most important to you. Leave what you want to see on Nomad PHP in the comments below and if we're able to we'll get it added to our roadmap!