Logging in PHP

How Logging works in Apache & PHP

Apache and php maintain the logs in the error.log file which is the default file for logging. 

File path to error.log is /var/log/apache2/error.log

PHP allows us to log into a different file using 

ini_set(‘error_log’, ‘path/to/log/file.log’)

Following are different methods for logging available

• error_log()
• trigger_error()
• syslog()

Here's a PHP function to create period wise logs

<?php class ErrorLog {    private $logFilePath;    /**     * ErrorLog constructor.     *     * @param string $logFilePath     */    public function __construct(string $logFilePath) {        $this->logFilePath = $logFilePath;    }    /**     * Parses the PHP error log to an array.     *     * @return \Generator     */    public function getParsedLogFile($sLogType = null, $dStartDate = null, $dEndDate = null): \Generator {        $parsedLogs = [];        $logFileHandle = fopen($this->logFilePath, 'rb');        while (!feof($logFileHandle)) {            $currentLine = str_replace(PHP_EOL, '', fgets($logFileHandle));            // Normal error log line starts with the date & time in []            if ('[' === $currentLine[0]) {                if (10000 === \count($parsedLogs)) {                    yield $parsedLogs;                    $parsedLogs = [];                }                // Get the datetime when the error occurred and convert it to berlin timezone                try {                    $dateArr = [];                    preg_match('~^\[(.*?)\]~', $currentLine, $dateArr);                    $currentLine = str_replace($dateArr[0], '', $currentLine);                    $currentLine = trim($currentLine);                    $dErrorDateTime = new DateTime($dateArr[1]);                    $dErrorDateTime->setTimezone(new DateTimeZone('Asia/Kolkata'));                    $errorDateTime = $dErrorDateTime->format('Y-m-d H:i:s');                } catch (\Exception $e) {                    $errorDateTime = '';                }                // Get the type of the error                if (false !== strpos($currentLine, 'PHP Warning')) {                    $currentLine = str_replace('PHP Warning:', '', $currentLine);                    $currentLine = trim($currentLine);                    $errorType = 'WARNING';                } else if (false !== strpos($currentLine, 'PHP Notice')) {                    $currentLine = str_replace('PHP Notice:', '', $currentLine);                    $currentLine = trim($currentLine);                    $errorType = 'NOTICE';                } else if (false !== strpos($currentLine, 'PHP Fatal error')) {                    $currentLine = str_replace('PHP Fatal error:', '', $currentLine);                    $currentLine = trim($currentLine);                    $errorType = 'FATAL';                } else if (false !== strpos($currentLine, 'PHP Parse error')) {                    $currentLine = str_replace('PHP Parse error:', '', $currentLine);                    $currentLine = trim($currentLine);                    $errorType = 'SYNTAX';                } else if (false !== strpos($currentLine, 'PHP Exception')) {                    $currentLine = str_replace('PHP Exception:', '', $currentLine);                    $currentLine = trim($currentLine);                    $errorType = 'EXCEPTION';                } else {                    $errorType = 'UNKNOWN';                }                if (false !== strpos($currentLine, ' on line ')) {                    $errorLine = explode(' on line ', $currentLine);                    $errorLine = trim($errorLine[1]);                    $currentLine = str_replace(' on line ' . $errorLine, '', $currentLine);                } else {                    $errorLine = substr($currentLine, strrpos($currentLine, ':') + 1);                    $currentLine = str_replace(':' . $errorLine, '', $currentLine);                }                $errorFile = explode(' in /', $currentLine);                $errorFile = '/' . trim($errorFile[1]);                $currentLine = str_replace(' in ' . $errorFile, '', $currentLine);                // The message of the error                $errorMessage = trim($currentLine);                if ($sLogType != null && $sLogType != $errorType) {                    continue;                }                echo "<pre>";                if ($dStartDate != null && $dStartDate > $dErrorDateTime) {                    continue;                }                if ($dEndDate != null && $dEndDate < $dErrorDateTime) {                    continue;                }                $parsedLogs[] = [                    'dateTime'   => $errorDateTime,                    'type'       => $errorType,                    'file'       => $errorFile,                    'line'       => (int)$errorLine,                    'message'    => $errorMessage,                    'stackTrace' => []                ];            } // Stack trace beginning line            else if ('Stack trace:' === $currentLine) {                $stackTraceLineNumber = 0;                while (!feof($logFileHandle)) {                    $currentLine = str_replace(PHP_EOL, '', fgets($logFileHandle));                    // If the current line is a stack trace line                    if ('#' === $currentLine[0]) {                        $parsedLogsLastKey = key($parsedLogs);                        $currentLine = str_replace('#' . $stackTraceLineNumber, '', $currentLine);                        $parsedLogs[$parsedLogsLastKey]['stackTrace'][] = trim($currentLine);                        $stackTraceLineNumber++;                    } // If the current line is the last stack trace ('thrown in...')                    else {                        break;                    }                }            }        }        yield $parsedLogs;    } } $oErrorLog = new ErrorLog(ini_get('error_log')); foreach ($oErrorLog->getParsedLogFile('SYNTAX',new DateTime('2021-04-25 12:15:30'), new DateTime('2021-04-25 16:00:05') ) as $key => $value) {    var_dump($value); }





Ref: From Internal memo provided by Kaveri Bhumkar

Check the amount of free disk space in Ubuntu

df -h --total

Using scp to copy files and folders

SCP (secure copy command) allows secure transferring of files between two hosts( does not matter if its local to local, local to remote, remote to local or remote to remote).

SCP uses the same authentication and security as the SSH protocol on which it is based. Since SCP relies on SSH (Secure Shell) to transfer files, all you need is the username and password for the source and target systems. 

LOCAL TO REMOTE

Copy a single file from the local machine to a remote machine:

The scp command needs a source and destination to copy files from one location to another location.

scp localmachine/path_to_the_file username@server_ip:/path_to_remote_directory

It will ask you to provide the password for that user, and then copy the file securely. 

Copy a local directory to a remote server:

If you want to copy the entire local directory to the server, then you use the -r (recursive) flag

scp -r localmachine/path_to_the_directory username@server_ip:/path_to_remote_directory/

Make sure that the source directory doesn’t have a forward slash at the end of the path, at the same time the destination path *must* have a forward slash.

Copy all files in a local directory to a remote directory

If you want to copy only the files inside a local directory to a remote directory, just add a forward slash(/) and * at the end of the source directory and give the path of destination directory. You will use the -r flag

scp -r localmachine/path_to_the_directory/* username@server_ip:/path_to_remote_directory/

 

REMOTE TO LOCAL

If you want to copy a single file, a directory or all files on a remote server to your local machine,the syntax's are the same as those shown above

Copy a single file:

scp username@server_ip:/path_to_remote_directory local_machine/path_to_the_file 

Copy a remote directory to a local machine:

scp -r username@server_ip:/path_to_remote_directory local-machine/path_to_the_directory/

Copy all files in a remote directory to a local directory:

scp -r username@server_ip:/path_to_remote_directory/* local-machine/path_to_the_directory/ 

 

REMOTE TO ANOTHER REMOTE

Copy a single file:

scp username@server1_ip:/path_to_the_remote_file username@server2_ip:/
  path_to_destination_directory/

Copy a directory from one location on a remote server to different location on the same server:

scp username@server1_ip:/path_to_the_remote_file username@server2_ip:/
  path_to_destination_directory/

Copy all files in a remote directory to a local directory

scp -r username@server1_ip:/path_to_source_directory/* username@server2_ip:/
  path_to_the_destination_directory/ 

REMOTE TO THE SAME REMOTE SERVER

Copy a single file:

scp username@server_ip:/path_to_the_remote_file username@server_ip:/
  path_to_destination_directory/

Copy a directory from one location on remote server to different location on the same server:

scp username@server_ip:/path_to_the_remote_file username@server_ip:/
  path_to_destination_directory/

Copy all files in a remote directory to a local directory

scp -r username@server_ip:/path_to_source_directory/* usdername@server_ip:/
  path_to_the_destination_directory/ 

Using curl to fetch from a URL which outputs a dynamically generated Excel file

We often come across this case where another company provides us with a url which we can  use via a browser to get an excel or pdf report. 

Usually it also accepts one or more parameters, which are used by the backend script in the report generation process.

How does one call this via CURL to allow our application to fetch the reports automatically  based on some pre defined schedule, or in a bulk fashion for a set of running parameters.

Its possible and to do it  one can use the delimiters -o and -j

Lets say our url looks something like this:

http://sample.example.com/xyz?type=xlsx&o=1&rt=1&id=11887&user=1

 Then we can use the call below to achieve this.

curl -o -j "http://sample.example.com/xyz?type=xlsx&o=1&rt=1&id=11887&user=1"

Apache Error notifier using Gulp

I wanted an easy way to retrieve critical errors from my Apache error log. While i was finding a solution, Gulp came into the picture. First let me introduce you to Gulp.

Gulp is a build system which you can use to automate your tasks. Its same as Grunt if you have heard of it.

I have used Gulp to automate following tasks in past:

1. Compiling SCSS 
2. Compressing CSS
3. Auto reloading my browser if CSS changes.
4. Compiling Coffeescripts

Now it was time to use gulp for Apache notification. So I created Gulp task for the same. Here is the script.

First of all I have included 3 Gulp dependencies which is Gulp itself as i don't have gulp in my environment, Gulp notifier to display error notification and Gulp Intercept to Go through log file and retrieve the error.

I have defined Watch as default task. Default task is a task which Gulp keep running continuously. The task keep watching over the log file. Whenever the error log file changes, Task will go through the file and will retrieve recent errors and will display them.

The script will display mainly 2 types of critical errors:

1. PHP Fatal Error
2. PHP Parse Error

Setup:

Requirement:

1. Node

Installation Steps:

1. First setup Node.
2. Install Gulp
3. Then clone this repository. Or Download repository from Github.
4. Run npm install

Start:

1. Goto the cloned repository.
2. First add path to your Apache's Php Error log file in variable "logPath". (Default is /var/log/apache2/error.log)
3. Run sudo gulp

How to Fix "PCLZIP_ERR_READ_OPEN_FAIL (-2) : Unable to open temporary file" Error?

If you have ever needed to ZIP files from your application, I guess, you must have heard of or used PCLZIP library. It is very simple, easy to use and feature-rich library.

If you have ever used this library on Linux, you might have come across this error stating PCLZIP_ERR_READ_OPEN_FAIL (-2) : Unable to open temporary file.

This error occurs because PCLZIP is using your current working directory to create the temporary file. But on Linux, Apache user may not have permission to write files in the current working directory so it will throw this error.

You can either give writing permission to your working directory, which I won't recommend, or you can set a temporary directory for PCLZIP.

To set temporary directory for PCLZIP, add following line to your code before you include PCLZIP Library:

define('PCLZIP_TEMPORARY_DIR', '/tmp/');

Or you can also use the path of a directory which is writable by your web server.

pre-commit Hook to Prevent Debug Value / Code From Being Committed

It is not uncommon that sometimes you end up committing a wrong code, probably with debug values and codes that you have put for temporary basis to test out something. I have several times committed a code that had some hardcoded values or debug messages which I put to test my code in different scenarios. It is quite hard to spot this mistake if your "wrong code" is the part of a rare scenario. And when this code goes into the production and the rare time comes, you will see either scared users or screaming users.

To avoid such mistakes, I wrote a small bash script few weeks back to be executed in pre-commit hook of git. This script looks for a particular text ( I call it #DUMMYTEXT) in your code in your staging area and stops you from committing the code if found. Unfortunately, you have to make habit of putting your #DUMMYTEXT every time (or at least once in the file) you are putting dummy values or code.

What is a Git Hook?

Hooks are nothing but a some scripts that are automatically executed for certain events.  To read more about hooks, go to http://git-scm.com/book/en/Customizing-Git-Git-Hooks

In our case, we will be using pre-commit, a client side hook, which is executed just before the code is committed. You can do whatever you want to do and decide if you want to proceed with commit or abort the commit. 

How do I create a hook?

In Git, hooks are per-repo basis. That means, you create separate hooks for each repository. You will find .git ( hidden) directory in root your repository. Inside that directory, you will see a .hooks directory. Usually, it is filled with sample hooks. In our case, we will need to put our script in pre-commit file. You can either create new one or rename pre-commit.sample to pre-commit and put the validation code/script there.

Where is the code?

#PUT YOUR OWN WORD TO LOOK FOR. BUT KEEP IT UNIQUE
debugText='#DUMMYTEXT'

filelist=(`git diff --cached --name-only --diff-filter=ACMR`)

found=0

for FILE in ${filelist[*]}
do
 if git grep --cached -q "$debugText" "$FILE"; then
  
  if [ $found -eq 0 ]; then
   echo "[ERROR] Following file(s) contains $debugText. Please remove debug text and values(if any):"
  fi
  found=1
  echo "$FILE"
  
 fi
  
done

exit $found

How do I work with this?

I recommend making a habit to put debug text every time you are putting dummy texts or values. So that you do not miss any dummy values. 

So if any time, you mistakenly stage a file with dummy text/values and try to commit , this script will stop you from committing the code.

Notes

If you forget to put your debug text, there is no way anyone case save you. ;-)  Feedback/Suggestion is highly appreciated! :-)