[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
On Wed, 5 Jun 2019 at 19:00, Matthew Kerwin <matthew at kerwin.net.au> wrote:
> We've put a lot of energy into this problem over the years. I've
> variously: overridden the default Search generation for some
> problematic views with custom SQL queries, and added a wrapper around
> the generate_views script that runs four instances in parallel (one
> for each core on our server). It still takes a couple of hours every
> week, but it's no longer 12+ hours.
> I'm not at work right now to get my hands on the code. I'll see if I
> can get to it tomorrow.
For those playing along at home, here's the script we use to run the
generation in parallel:
The parallelisation is a bit magical, and requires you to specify the
`MAX_PARALLEL_CHILDREN=x` environment variable. I've tried to
sanitise it of all our particular quirks, but might have missed