Log in

View Full Version : PHP's memory usage?


Llort
19-01-2010, 02:08 PM
Lo tech-people,

I am busy implementing some image-color-recognition for my work. All was working fine (written it in JAVA first). Just implemented in PHP.

First of all speed. Damn! With JAVA it takes about 1second to read a 160*160 image and do some color recognition (mozaicify, LUT+RLE, blob finding).
Takes around 14seconds on the same machine, but running under PHP.

So we wanted to see how long it took on a live server. Blank page... *enables errors*. Memory exhausted? 32MB? !!
So I started to output memory usage on my local machine and it tops 59MB! for a 160x160 image!.

So I started to debug some more.
Found out that after reading all the pixel's in an array the memory usage is around 20MB
20MB! For an array 160x160 each field holding an other array(H,S,L)
and an other 160x160 array each field holding array(R,G,B)

How is this possible? Before I start reading the pixels, memory usage is around 1.25MB...

11:1.25
15:1.25
19:1.25
23:1.25
27:1.5 << before
38:20.25 >>after reading pixels + storing
42:20.25 >>after imagedestroy(); (to release the GD2 image)
46:20.25 >>after I unset the string of data that imagecreatefromstring() needed

The number before the : is the line number. Number after the : is memory usage in MB


private function setHSV($x,$y,array $hsv){
if(!is_array($this->dataHSV[$x]))
$this->dataHSV[$x] = array();
// $this->dataHSV[$x] = array_fill(0,$this->getHeight(),0);
$this->dataHSV[$x][$y] = $hsv;
}

private function setRGB($x,$y,array $rgb){
if(!is_array($this->dataRGB[$x]))
$this->dataRGB[$x] = array();
// $this->dataRGB[$x] = array_fill(0,$this->getHeight(),0);
$this->dataRGB[$x][$y] = $rgb;
}

Ruoste
19-01-2010, 04:21 PM
I did a quick test with the setHSV function and indeed with a multi-dimensional 160 * 160 * 3 array it used approximately 10MB of memory.

PHP is not very memory efficient with variables or arrays. If you store an integer or just a pointer to another array, php ends up using a lot of extra space for various information about it.

You could save a chunk of memory by using a one-dimensional array instead. With one dimensional array where each index had the $hsv array the memory usage went down to around 6MB. Storing each $hsv array in one integer cut another 2/3 out of the memory usage, down to around 2.3MB

Might not be ideal still but could be a solution if you have to use php. You almost definitely won't be able to make it go nearly as fast as the Java version.

Llort
19-01-2010, 07:22 PM
yeah, we've been concidering altering the test-java-app to do all the work. But that would mean setting up a PC at work or something.

Anyway, for some reason, even initializing the array's in this fashion:


$dataRGB = array_fill(0,160,array_fill(0,160,array(0,0,0)));


Wouldnd really increment memory usage. (stayed below 1.5MB).
As soon as I altered the $dataRGB after init the memory usage skyrocketed. to said 10MB.
even if just doing

for($x=0;$x<160;$x++){
for($y=0;$y<160;$y++){
$dataRGB[$x][$y] = array(rand(0,255),rand(0,255),rand(0,255));
}
}


Would make the memory usage go up to 10MB...
And it would only alter the data not add anything new or something... :S

Llort
25-01-2010, 09:27 AM
meh,

for some reason, php is really strange on memory management....

<?php
$size = 100;
echo "(".__LINE__.")".memory_get_usage()."<br />\n";
$array = array_fill(0,$size,array_fill(0,$size,array(0,0,0) ));

foreach($array AS $key1 => $sub1){
foreach($sub1 AS $key2 => $sub2){
echo "(".__LINE__.")".memory_get_usage()."(".$sub2[0].")<br />\n";
}
}
this, will work just fine. Output:
(3)326720
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)
(8)342432(0)

Changing the foreach from copy's to references like this:
<?php
$size = 100;
echo "(".__LINE__.")".memory_get_usage()."<br />\n";
$array = array_fill(0,$size,array_fill(0,$size,array(0,0,0) ));

foreach($array AS $key1 => &$sub1){
foreach($sub1 AS $key2 => &$sub2){
echo "(".__LINE__.")".memory_get_usage()."(".$sub2[0].")<br />\n";
}
}
Will result in this:
(3)326720
(8)342680(0)
(8)342928(0)
(8)343176(0)
(8)343424(0)
(8)343672(0)
(8)343920(0)
(8)344168(0)
(8)344416(0)
(8)344664(0)
..... snip.....
(8)3309836(0)
(8)3310084(0)
(8)3310332(0)
(8)3310580(0)
(8)3310828(0)
(8)3311076(0)
(8)3311324(0)
(8)3311572(0)
(8)3311820(0)
(8)3311820(0)

Doing either of those but, instead of reading, writing new values to $sub2 or $array[$key1][$key2] will result in the same memory increase as in the second example....

I am running a PHP5.3 server btw.

Psonica
28-01-2010, 05:12 PM
PHP is funky when it comes to memory consumption ... but who nowdays limits their PHP to take just 32MB when the average server clocks in at 4GB+? RAM is cheaper than optimizing code (sadly) so...

Also lack of typecasting variables probaby makes PHP assume 64bit values instead of 8bit?

<nostalgia>
I remember when we were limited to 8bit processors running at (almost) 1MHz and a whoppin' 64kB of RAM ... those were the times.
</nostalgia>

...I fail at providing proper solutions, I must get back to programming again.

Llort
28-01-2010, 05:25 PM
You remember using them? I use them Well, not microprocessors, but microcontrollers. (Builtin flash/ram). Fastest one (in my possession and running) atm clocks at 80MHZ resulting in 40MIPS, 16bits. RISC Microcontroller. 32kbyte RAM, 128kbyte flash. 4kbyte DMA memory space shared with the 32kbyte RAM.
16kbyte X/Y RAM for DSP co-processor (shared with the 32kbyte RAM).
And I made it in to an image processor. Attached to a 10FPS 640x480 CMOS camera. 8-bits data interface. Controller is capable of tracking at 10FPS for 8objects per frame. Recognized by colour.
Camera interface written in assembly. Tracking code written in C.
Total RAM usage? 6kbyte, total flash? 2kbyte.
(We need the extra space because we will also be building voice-recognizing in to it, hence we need the DSP version of it) :P

slowest microcontroller I am currently using is an 8bits 4MIPS (16mhz) 256bytes of ram, 512bytes of flash. Doing random things (For testing C/assembly code :P)

[/offtopic]

I've reduced memory usage for my PHP program from 52MB to ~14MB. Total image processing time from 14secs per image to ~>2secs per image.