2020-12-09 09:29

Uberbank workflow

Hi all,

This is the rest of the changes to implement an uberbank workflow in pegasus. To summarize: This, coupled with #986 #976 and #973, implements the following changes:

GEOM BANK - The geometric bank workflow is moved over from glue.pipeline to pycbc.workflow. (#973) - The geometric bank workflow is edited to use HDF files (and many less files) for internal data products (output is still XML in standard format) (#986) - The geometric bank codes are optimized with some numpy-isation (#986) - Support for running the geom bank as a sub-workflow is added (here) - The general structure of the geom_bank workflow, and scientific input choices, are unchanged. One should still be able to reproduce previous output with the new code.

SBANK - A new sbank workflow generator is created using pycbc.workflow, intended to deprecate the version in lalapps (#976). - This workflow uses recent changes in sbank to allow for more flexibility in the workflow (#976 and here) - Specifically we now have the concept of multiple parallel stages allowing the bank to built up over time, while the user can monitor the process, the chirp-mass binning is also updated each cycle allowing for more efficiency - A "readder" stage is added to ensure the parallel jobs do not place templates at the same point in parameter space. - Support is added to allow the sbank workflow generator to run as a sub-workflow as well as a standalone workflow. (here)

UBERBANK (all changes here) - A, let's call it "first draft", of the uberbank workflow is added, demonstrating how to tie sbank and geom_bank workflows together. - This implementation runs 1 sbank workflow and 1 geom_bank workflow independently at first, and then runs 1 further sbank workflow using input from the original two workflows as a seed bank. - It would be good to add more flexibility in the structure, but this makes a start, and we can add functionality as needed. - Handling of sub-daxes in pycbc.workflow is not as neat as normal files. We might want to consider extending upon the Dax class in workflow.core to make some of the stuff here a little more automatic.

CONFIG FILES - I'm still working an optimal config files for this. Especially sbank now has a bunch of knobs and dials to turn, which we want to do to balance efficiency with coverage. - For now I attach a short example ini file, which is not intended to produce a decent template bank, but is intended to demonstrate how this works.


At the moment it would be good to have some people try to run this, I'll try and provide a more meaningful ini file, and let me know if there are problems/suggestions etc. Please feel free to open pull requests against this branch, if you want to include functionality in this patch.


  • 点赞
  • 回答
  • 收藏
  • 复制链接分享