Question regarding Higher-Order-Programming in Python

Question regarding Higher-Order-Programming in Python

Post by Mark Fin » Thu, 23 Dec 2010 17:50:04


I am about to learn Higher-Order-Programming with Lisp, Haskell, and
Python. Some people who have gone completely out of their mind call
this FP.

In Haskell I learned that when I use map on a list it starts nesting
as soon as I start adding elements. If I do not like the nesting I use
ConcatMap.

In Python I have a similar scenario. I have a generator which creates
some combinatorics of a input dictionary. The list starts nesting. Now
I could use reduce(concat, ...) which would be the correct thing from
a functional perspective. But from a resource utilization perspective
it is not a good idea since the reduce is executing the generator.

I tried to illustrate this using a small example (the often in
combinatorics the real thing would be much bigger that is why I need
to use a generator):

{'special': ['+', '-'], 'number': [1, 2, 3], 'letter': ['a', 'b']}
... # helper to get products from keys in the following form:
... # [('bold', True), ('color', 'black')]
... values = itemgetter(*keys)(dims)
... product = it.product(*values)
... return map(partial(zip, keys), product)
...
[('special', 'number'), ('special', 'letter'), ('number', 'letter')]
[[[('special', '+'), ('number', 1)], [('special', '+'), ('number',
2)], [('special', '+'), ('number', 3)], [('special', '-'), ('number',
1)], [('special', '-'), ('number', 2)], [('special', '-'), ('number',
3)]], [[('special', '+'), ('letter', 'a')], [('special', '+'),
('letter', 'b')], [('special', '-'), ('letter', 'a')], [('special',
'-'), ('letter', 'b')]], [[('number', 1), ('letter', 'a')],
[('number', 1), ('letter', 'b')], [('number', 2), ('letter', 'a')],
[('number', 2), ('letter', 'b')], [('number', 3), ('letter', 'a')],
[('number', 3), ('letter', 'b')]]] # the resulting list is nested one
level to deep caused by the map(get_products, ..

My problem is that I want to get single elements from the generator
like [('special', '+'), ('number', 1)]. But this does not work because
the list is now nested to deep.

That is what I expect: (I could get something like that with the
following >>> res = reduce(concat, res)
[[('special', '+'), ('number', 1)], [('special', '+'), ('number', 2)],
[('special', '+'), ('number', 3)], [('special', '-'), ('number', 1)],
[('special', '-'), ('number', 2)], [('special', '-'), ('number', 3)],
[('special', '+'), ('letter', 'a')], [('special', '+'), ('letter',
'b')], [('special', '-'), ('letter', 'a')], [('special', '-'),
('letter', 'b')], [('number', 1), ('letter', 'a')], [('number', 1),
('letter', 'b')], [('number', 2), ('letter', 'a')], [('number', 2),
('letter', 'b')], [('number', 3), ('letter', 'a')], [('number', 3),
('letter', 'b')]]

I have seen the problem many times but so far I could not google a
solution on the web.

By the way do you know any substantial example using FP in Python
(generators, imap, ifilter, ...)?
 
 
 

Question regarding Higher-Order-Programming in Python

Post by Peter Otte » Thu, 23 Dec 2010 21:57:02


Like this?

[('special', '+'), ('special', '-'), ('number', 1), ('number', 2),
('number', 3), ('letter', 'a'), ('letter', 'b')]

But these are just glorified for-loops, so:

dims.itervalues()))))
[('special', '+'), ('special', '-'), ('number', 1), ('number', 2),
('number', 3), ('letter', 'a'), ('letter', 'b')]

Peter

 
 
 

Question regarding Higher-Order-Programming in Python

Post by Mark Fin » Thu, 23 Dec 2010 23:35:26

> >>> list(chain.from_iterable(starmap(product, izip(izip(dims.iterkeys()),

so far I have never noticed chain.from_iterable, but many thanks to
you Peter, I have now a beautiful solution to this problem.
[('special', '+'), ('number', 1)]
[('special', '+'), ('number', 2)]
 
 
 

Question regarding Higher-Order-Programming in Python

Post by Arnaud Del » Fri, 24 Dec 2010 06:45:12

Mark Fink < XXXX@XXXXX.COM > writes:


You can also write this as:

l = (p for c in comb for p in get_products(c))


Also in your original post you define get_products:


You could define it as e.g.

def get_products(keys):
key_values = [[(k, v) for v in values] for k in keys]
return it.product(*key_values)

But maybe for some reason you are trying to avoid list comprehensions
and generator expressions?

--
Arnaud