In Python, Is There A Way To Extract A Embedded Json String?

So I'm parsing a really big log file with some embedded json. So I'll see lines like this foo='{my_object:foo, bar:baz}' a=b c=d The problem is that since the internal json can h

Solution 1:

Found a really cool answer. Use json.JSONDecoder's scan_once function

In [30]: import json

In [31]: d = json.JSONDecoder()

In [32]: my_string = 'key="{"foo":"bar"}"more_gibberish'

In [33]: d.scan_once(my_string, 5)
Out[33]: ({u'foo': u'bar'}, 18)

In [37]: my_string[18:]
Out[37]: '"more_gibberish'

Just be careful

In [38]: d.scan_once(my_string, 6)
Out[38]: (u'foo', 11)

Solution 2:

Match everything around it.

>>> re.search('^foo="(.*)" a=.+ c=.+$', 'foo="{my_object:foo, bar:baz}" a=b c=d').group(1)
'{my_object:foo, bar:baz}'

Solution 3:

Use shlex and json.

Something like:

import shlex
import json

def decode_line(line):
    decoded = {}
    fields = shlex.split(line)
    for f in fields:
        k, v = f.split('=', 1)
        if k == "foo":
            v = json.loads(v)
        decoded[k] = v
    return decoded

This does assume that the JSON inside the quotes is quoted properly.

Here's a short example program that uses the above:

import pipes

testdict = {"hello": "world", "foo": "bar"}
line = 'foo=' + pipes.quote(json.dumps(testdict)) + ' a=b c=d'print line
print decode_line(line)

With output:

foo='{"foo": "bar", "hello": "world"}' a=b c=d
{'a': 'b', 'c': 'd', 'foo': {u'foo': u'bar', u'hello': u'world'}}