Re: [PLUG] Python nested dict data structure

[Victor via plug <plug@lists.phillylinux.org>]

On Fri, Jul 2, 2021 at 3:51 PM JP Vossen via plug
<plug@lists.phillylinux.org> wrote:
>
> If I want to read multiple files, and build a data structure like this by adding different keys and values at different times, and/or incrementing an existing numerical value:
> ```
> {'Acme Inc': {'region': 'US', 'counter': 3}}
> ```
>
> In Perl, but using Pythonic syntax, this would auto-instantiate and Just Work:
> ```
> dict = {}
> company = 'Acme Inc'
> dict[company]['region']   = 'US'
> dict[company]['counter'] += ave
> ```
>
> Clean, and simple...except that Perl's syntax for nested data structures is virtually unreadable and really hard to write, for me at least.  Overall I'm liking Python, but there are some things, like this, that are hurting my head.
>
> In Python it's really easy to create that structure at one time, but that is NOT how most of my use-cases work.  I'm always pulling fields out of 2-3 different files and building a data structure in pieces over time.  Like company and region are in 1 file, and a whole bunch of records containing something I want to count are in another file(s), with the 'company' key to link them.
>
> This works in Python 3 and 2 (sad to say I still have old nodes with only Python2):
> ```
> #!/usr/bin/env python3
> # dict.py--do I REALLY have to do all this crap just for NESTED dicts!?
>
> dict = {}
>
> # Add Nested Key/Value pair to dict
> def add_nstkv(dict, key, subkey, value):
>      if key not in dict:         # Key must exist or error
>          dict[key] = {}          # Create empty sub-dict
>      dict[key][subkey] = value   # Add value
>
> # Add/inc_nstkv a value in a nested dict
> def inc_nstkv(dict, key, subkey, value):
>      if key not in dict:              # Parent key must exist or error
>          dict[key] = {}               # Create empty sub-dict
>      if subkey not in dict[key]:      # Subkey must exist or error
>          dict[key][subkey]  = value   # Add value
>      else:
>          dict[key][subkey] += value   # inc_nstkv
>
> company = 'Acme Inc'
> add_nstkv(dict, company, 'region', 'US') # Does: dict[company]['region']   = val
> inc_nstkv(dict, company, 'counter', 1)   # Does: dict[company]['counter'] += val
> inc_nstkv(dict, company, 'counter', 2)
> print(dict)
> ```
>
> Output:
> ```
> $ ./dict.py
> {'Acme Inc': {'region': 'US', 'counter': 3}}
> ```
>
> I've tried `dict = defaultdict()` and `dict = defaultdict(int)` but both fail since I have mixed string and int values.  I've tried all kinds of other crazy stuff and nothing else worked either.  Most Google results are too-trivial `d[k] = v` and don't help with nested structs.
>
> Do I *really* have to write all that extra code every time I want to do something this simple?  Or am I missing something?
>
> Later,
> JP
> --  -------------------------------------------------------------------
> JP Vossen, CISSP | http://www.jpsdomain.org/ | http://bashcookbook.com/
> ___________________________________________________________________________
> Philadelphia Linux Users Group         --        http://www.phillylinux.org
> Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
> General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug

Using the dictionary methods .setdefault() and .get() [1] is much more
pythonic, but it is confusing the first few times you are mangling
dictionaries like this. Merge and Update using | and |=, respectively,
are new in 3.8 (I think) and worth playing with. I haven't gotten to
use either of those much yet, so I don't know that they apply in your
situation.

```
#!/usr/bin/env python3
# d.py--do I REALLY have to do all this crap just for NESTED ds!?

# dict is reserved. it works, but I avoid using it.
d = {}

# Add Nested Key/Value pair to d
def add_nstkv(d, key, subkey, value):
    d.setdefault(key, dict())
    d[key][subkey] = value   # Add value

# Add/inc_nstkv a value in a nested d
def inc_nstkv(d, key, subkey, value):
    d.setdefault(key, dict())
    d[subkey] = d.get(subkey, 1) + 1

company = 'Acme Inc'
add_nstkv(d, company, 'region', 'US') # Does: d[company]['region']   = val
inc_nstkv(d, company, 'counter', 1)   # Does: d[company]['counter'] += val
inc_nstkv(d, company, 'counter', 2)
print(d)
```

[1] https://docs.python.org/3/library/stdtypes.html#typesmapping
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug