Parsing a Large JSON Object in PHP Using Less Memory

I've got a small project where I need to store between 500 and 1500 product records. I've thought about using MySQL, but I'd like to keep this particular application completely self contained (no real database). I'd also like to be able to edit the “database” very quickly.

One option I'm considering is storing the data in JSON format. But, PHP arrays take a lot of memory and the common way to load JSON data is to load it into an array.

When I load that entire JSON object from file it takes about 75k as a string. That's a reasonable and acceptable amount of memory for my particular app.

When I json_decode() that string to an array, it takes almost 18MB. There are lots of reasons for this, but PHP optimizes array's for speed instead of memory. Therefore a big array like this will consume a lot of memory. This is more memory than I'd like to consume for this particular application.

One thought I've had is loading the JSON data into a string and then using PHP's regular expression functions to parse out the bit I need (probably based on a key in the JSON data) and convert only that one item to an array. That should save a lot of RAM in the app.

I could also skip converting to an array at all and pass the entire JSON object to the client side where I could process it with Javascript. I don't have to worry as much about memory on the client side. I do have to worry about how much data I transmit to the client though. In this case, about 75k of data each time the client opens my web based app. That might or might not be too much for the client to handle.

My app requires dozens of pictures. Even if they are optimized, those pictures are about 50k each. That means that my data is about the size of a single image download in my app.

comments powered by Disqus
php/parsing_a_large_json_object_in_php_using_less_memory.txt · Last modified: 2020/06/01 22:53 (external edit)