One of our server is filled with coredump errors on apache Error Logs. When I started exploring this coredump problem, then I come to know that I need to do few more extra step to start analysing this apache2 core dump issue. I tried almost all tutorials which I found online to enable this coredump logging on apache server. But most articles are outdated, and some tips don’t work for me to generate coredump files on our server. If you are also facing such issue, then here is a quick guide to enable coredump file on apache ubuntu server.

I was using Apache2 server and Ubuntu Server Edition Operating System.

In our server Apache error log, I can see many apache coredump errors. Below are the few coredump error which I got on my server.

root@servername:~# tail -300 /var/log/apache2/error.log

exit signal Segmentation fault (11), possible coredump in /etc/apache

[Mon Feb 29 12:26:28.079037 2016] [core:notice] [pid 1384] AH00051: child pid 13318 exit signal Segmentation fault (11), possible coredump in /etc/apache2
[Mon Feb 29 12:47:04.374703 2016] [core:notice] [pid 1384] AH00051: child pid 14037 exit signal Segmentation fault (11), possible coredump in /etc/apache2
[Mon Feb 29 12:48:02.460749 2016] [core:notice] [pid 1384] AH00051: child pid 13924 exit signal Segmentation fault (11), possible coredump in /etc/apache2

To know why this apache2 is crashing, first we need to enable coredump log generation on server so that we can analyze what process causing apache to crash our server.

I was also trying to enable CoreDump log generation by following some online tutorials which I found. Unfortunately, most of the resource which I found are an outdated tutorial. After researching little more, I got a small clue. I followed that, and it worked for me.

If you are also facing the issue in generating coredump file on apache2 server on ubuntu operating system, then give it a try once for this method.

Enable CoreDump Log Generation on Apache2, Ubuntu Server:

1. For latest apache2 version, you find the official guide in the below server directory /usr/share/doc/apache2/README.backtrace

2. You can view the content of that file by executing below command
3. root@servername:~# cat /usr/share/doc/apache2/README.backtrace

Below is the content you will get on that file.

If apache crashes or freezes, it is helpful if you include a backtrace in the bug report.In case of a crash, do the following:

1) Install the packages apache2-dbg libapr1-dbg libaprutil1-dbg gdb.

2) Add “CoreDumpDirectory /var/cache/apache2” to your apache configuration.

3) Execute as root:
/etc/init.d/apache2 stop
ulimit -c unlimited
/etc/init.d/apache2 start

4) Do whatever it takes to reproduce the crash. There should now be the file
/var/cache/apache2/core .

5) If you use a forking MPM (e.g. mod_prefork), execute:

gdb /usr/sbin/apache2 /var/cache/apache2/core
(gdb) bt full
...
(gdb) quit

If you use a threaded mpm (mod_worker, mod_event), execute:

gdb /usr/sbin/apache2 /var/cache/apache2/core
(gdb) thread apply all bt full
...
(gdb) quit

Include the backtrace in the bug report.

6) Undo the change to your configuration, uninstall the debug packages, remove
/var/cache/apache2/core

In case of a hanging process, you don’t need a core dump and you can skip steps 2 and 3.

4) Reproduce the problem. Get the pid of a hanging process.

5) Start gdb with

gdb -p pid

and continue as described above. It may also be helpful to include the output of

strace -p pid

 

5.Now run the below command
sudo apt-get install apache2-dbg libapr1-dbg libaprutil1-dbg gdb

6. Now open your apache2 config file
nano /etc/apache2/apache2.conf

7. At the end of the file, add the below line

CoreDumpDirectory /var/cache/apache2/

8. Now run the below command one by one

/etc/init.d/apache2 stop


ulimit -c unlimited


/etc/init.d/apache2 start

9. To give permission to that directory, execute below command one by one

chown -R www-data:www-data /var/cache/apache2

chmod 777 /var/cache/apache2

10. Now we need to manually generate core dump first by executing below command

kill -11 19122

19122 –> apache2 Process ID. You can get apache process Id by executing top command.

 

 

That’s it. Now onwards your apache2 will start generating coredump log files at /var/cache/apache2/core.  You can view core dump file by using

gdb /usr/sbin/apache2 /var/cache/apache2/core

I Hope this article help you guys in getting core dump file on apache. Don’t forget to say thanks in comments.