making uniformly distributed facets fails

[AHorneffer]

If I set minimize_nonuniformity = True in the parset, then factor fails with:

Traceback (most recent call last):
  File "/opt/soft/lofar-stuff/bin/runfactor", line 4, in <module>
    __import__('pkg_resources').run_script('FACTOR==1.0', 'runfactor')
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 534, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1438, in run_script
    execfile(script_filename, namespace, namespace)
  File "/.aux_mnt/lofard4a/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/EGG-INFO/scripts/runfactor", line 70, in <module>
    reset_operations)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/process.py", line 73, in run
    test_run, reset_directions, reset_operations)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/process.py", line 571, in _set_up_directions
    max_radius_deg=max_radius_deg, dry_run=dry_run)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/process.py", line 771, in _initialize_directions
    flux_min_for_merging_Jy=dir_parset['flux_min_for_merging_jy'])
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 347, in make_directions_file_from_skymodel_uniform
    groupSize,searchDepth = chooseGroupSize(directions_max_num,ncpu=ncpu,maxTime=maxTime)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 182, in chooseGroupSize
    resG,resN = chooseGroupSize(K,ncpu=ncpu,timeFactor=timeFactor,maxTime=maxTime,minGroupSize=minGroupSize - 1)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 182, in chooseGroupSize
    resG,resN = chooseGroupSize(K,ncpu=ncpu,timeFactor=timeFactor,maxTime=maxTime,minGroupSize=minGroupSize - 1)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 182, in chooseGroupSize
    resG,resN = chooseGroupSize(K,ncpu=ncpu,timeFactor=timeFactor,maxTime=maxTime,minGroupSize=minGroupSize - 1)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 182, in chooseGroupSize
    resG,resN = chooseGroupSize(K,ncpu=ncpu,timeFactor=timeFactor,maxTime=maxTime,minGroupSize=minGroupSize - 1)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 182, in chooseGroupSize
    resG,resN = chooseGroupSize(K,ncpu=ncpu,timeFactor=timeFactor,maxTime=maxTime,minGroupSize=minGroupSize - 1)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 169, in chooseGroupSize
    if (K % groupSize) < minGroupSize + n:#remainder will be less than 5 (so not good uniformity)
ZeroDivisionError: integer division or modulo by zero

I tracked it down to maxTime being set to 5 (minutes?) in make_directions_file_from_skymodel_uniform(). I.e. computeTime is always larger than maxTime in chooseGroupSize(), the minimum value of computeTime being 6.3.

Would it make sense to scale the default / starting value for maxTime by the number of CPUs?

[AHorneffer]

Well, if I change the default value of maxTime to something higher (I chose 50), the it get only a little further:

Traceback (most recent call last):
  File "/opt/soft/lofar-stuff/bin/runfactor", line 4, in <module>
    __import__('pkg_resources').run_script('FACTOR==1.0', 'runfactor')
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 534, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1438, in run_script
    execfile(script_filename, namespace, namespace)
  File "/.aux_mnt/lofard4a/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/EGG-INFO/scripts/runfactor", line 70, in <module>
    reset_operations)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/process.py", line 73, in run
    test_run, reset_directions, reset_operations)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/process.py", line 571, in _set_up_directions
    max_radius_deg=max_radius_deg, dry_run=dry_run)
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/process.py", line 771, in _initialize_directions
    flux_min_for_merging_Jy=dir_parset['flux_min_for_merging_jy'])
  File "/opt/soft/lofar-stuff/lib/python2.7/site-packages/FACTOR-1.0-py2.7.egg/factor/directions.py", line 362, in make_directions_file_from_skymodel_uniform
    combination = calibratorGroupCombinations.next()
StopIteration

P.S. the line-numbers in directions.py are changed by the debug messages that I added. But I guess the code lines are clear enough to find.

[Joshuaalbert]

As I made this, I must point out that it was not implemented properly and so it fails. I'm fixing it right now.

[Joshuaalbert]

Upon looking at this it looks like there is an issue with calculating how long the minimization should run for. I'm going to modify to run faster and remove the intermediate step. I'll also clean the code up to be more readable.

[Joshuaalbert]

It is taking longer than an afternoon to fix so I'm moving it down my priority list for now.