Auto-ROI

Details for developers and power-users

Introduction

This feature is currently available in the prerelease branch. A high-level overview and user instructions can be found here. The feature was originally developed in a standalone repository but has since been merged into BakingTray.

Getting this feature right is very important: if it makes mistakes there could be data loss. Accordingly, a library of 133 past acquisitions (many with multiple samples so in total there are over 300 samples) was used to test the algorithm. Almost all samples used were rat or mouse brains. Development of this project was highly test-oriented: changes to the code were always checked against a reference result set.

Generating pStack files

A pStack is a structure containing a preview image stack along with some extra information. These were used for developing the auto-ROI feature. For example, the command autoROI.test.runOnStackStruct(pStack) calculates bounding boxes for a whole acquisition. We can then evaluate if a good job was done and tweak the algorithm accordingly.

The input argument pStack is a structure which needs to be generated by the user. It's a good idea to generate these and store to disk in some reasonable way. e.g. Inside sub-directories divided up however makes sense, such as one directory containing all acquisitions of single samples, one with two samples, etc. To generate a pStack file do the following

We will work with imaging stacks (imStack, below) obtained from the BakingTray preview stacks.

>> nSamples=2;
>> pStack = autoROI.groundTruth.stackToGroundTruth(imStack,'/pathTo/recipeFile',nSamples)

pStack = 

  struct with fields:

               imStack: [1138x2826x192 int16]
                recipe: [1x1 struct]
    voxelSizeInMicrons: 8.1855
     tileSizeInMicrons: 1.0281e+03
              nSamples: 2
             binarized: []
               borders: {}

There are two empty fields (binarized and borders) in the pStack structure. These need to be populated with what we will treat as a proxy for ground truth: which regions actually contain brain. This is necessary for subsequent evaluation steps but is not necessary to run the automatic tissue-finding code. This is done with:

pStack=autoROI.groundTruth.genGroundTruthBorders(pStack,7)

And the results visualised with:

>> volView(pStack.imStack,[1,200],pStack.borders)

Correct any issues you see by any means necessary.

Generating bounding boxes from a stack structure

>> OUT=autoROI.test.runOnStackStruct(pStack)

Visualise it:

>> b={{OUT.roiStats.BoundingBoxes},{},{}}
>> volView(pStack.imStack,[1,200],b)

Evaluating results

Here we run the algorithm on all pStack files. First ensure you have run analyses on all samples. Run the test script on one directory:

>> autoROI.test.runOnAllInDir('stacks/singleBrains')

You can optionally generate a text file that sumarises the results:

>> autoROI.test.evaluateDir('tests/191211_1545')

To plot all samples:

autoROI.evaluate.plotResults('tests/191211_1545')

To visualise the outcome of one sample:

>> load LIC_003_previewStack.mat 
>> load tests/191211_1545/log_LIC_003_previewStack.mat
>> b={{testLog.BoundingBoxes},{},{}};
>> volView(pStack.imStack,[1,200],b);

To run on all directories containing sample data within the stacks sub-directory do:

>> autoROI.test.runOnAllInDir

Running re-runnin autoROI on particular section from a test acquisition

pStack.imStack(:,:,30:end)=[]; % We only care about the first few sections
OUT_S=autoROI.test.runOnStackStruct(pStack); % run on all

% We notice a problem around section 23, so let's run just that 

tmp_s=OUT_S;
tmp=pStack;
ind=23;
tmp.imStack = pStack.imStack(:,:,ind);
tmp_s.roiStats(ind:end)=[];
autoROI(tmp,'lastSectionStats',tmp_s,'showbinaryimages',true); % Shows outcome for this section only

How the auto-ROI works

The general idea is that bounding boxes around sample(s) are found in the current section (n), expanded by about 200 microns, then applied to section n+1. When section n+1 is imaged, the bounding boxes are re-calculated as before. This approach takes into account the fact that the imaged area of most samples changes during the acquisition. Because the acquisition is tiled and we round up to the nearest tile, we usually end up with a border of more than 200 microns. In practice, this avoids clipping the sample in cases where it gets larger quickly as we section through it. There is likely no need to search for cases where sample edges are clipped in order to add tiles. We image rectangular bounding boxes rather than oddly shaped tile patterns because in most cases our tile size is large.

Implementation

imStack is a downsampled stack that originates from the preview images of a BakingTray serial section 2p acquisition. To calculate the bounding boxes for section 11 we would run:

autoROI(imStack(:,:,10))

The function will return an image of section 10 with the bounding boxes drawn around it. It uses default values for a bunch of important parameters, such as pixel size. Of course in reality these bounding boxes will need to be evaluated with respect to section 11. To perform this exploration we can run the algorithm on the whole stack. To achieve this we load a "pStack" structure, as produced by autoROI.test.runOnStackStruct, above. Then, as described above, we can run:

 autoROI.test.runOnStackStruct(pStack)

How does autoROI actually give us back the bounding boxes when run the first time (i.e. not in a loop over a stack)? It does the following:

• Downsample the stack again to a fixed size: currently 50 microns.

• Median filter the stack with a 2D filter

• On the first section, derives a threshold between brain and no-brain by using the median plus a few SDs of the border pixels.

  We can do this because the border pixels will definitely contain no brain the first time around.

• On the first section we now binarize the image using the above threshold and do some morphological filtering to tidy it up and to expand the border by 200 microns. This is done by the internal function binarizeImage.

• This binarized image is now fed to the internal function getBoundingBoxes, which calls regionProps to return a bounding box.

  It also: removes very small boxes, provides a hackish fix for the missing corner tile, then sorts the bounding boxes in order of ascending size.

• Next we use the external function autoROI.mergeOverlapping to merge bounding boxes in cases where the is is appropriate. This function is currently problematic as it exhibits some odd behaviours that can cause very large overlaps between bounding boxes.

• Finally, bounding boxes are expanded to the nearest whole tile and the merging is re-done.

Making summaries

autoROI.test.evaluateBoundingBoxes works on a stats structure saved by autoROI.test.runOnAllInDir. We can do the whole test directory with autoROI.test.evaluateDir.

Unit tests

Testing revolves around ensuring that the output of the algorithm is unchanged (or improved) following modifications to the code. e.g. This can be used to check whether the current commit produces a result directory at least as good as the last good reference run. To test for this:

>> cd ./previewStacks
>> autoROI.test.runOnAllInDir('stacks')

This produces an output in previewStacks/tests. If this is the first time you are doing this and need a reference then output of the test should be moved to previewStacks/test_reference. So you have:

previewStacks/stacks
previewStacks/test_reference
previewStacks/tests

To look at the results, cd to the tests directory and run autoROI.evaluate.plotResults(PATH_TO_previewStacks). e.g.

>> autoROI.evaluate.plotResults('/Volumes/data/previewStacks/tests/210413_1441')

To compare to the reference stack:

autoROI.evaluate.compareResults('./test_reference','/Volumes/data/previewStacks/tests/210413_1441')

Example usage within BakingTray

Load a preview stack, "take" a preview image, find the threshold, calculate and display the bounding boxes.

hBT.scanner.attachPreviewStack(pStack);
% press "Preview Scan" in the GUI
hBT.getThreshold

In the API we find:

>> hBT.autoROI

ans = 

  struct with fields:

    previewImages: [1x1 struct]
            stats: [1x1 struct]

>> hBT.autoROI.previewImages

ans = 

  struct with fields:

               imStack: [962x570 double]
                recipe: [1x1 recipe]
    voxelSizeInMicrons: 20
     tileSizeInMicrons: 966.6333
              nSamples: []
               fullFOV: 1

>> hBT.autoROI.stats

ans = 

  struct with fields:

        origPixelSize: 20
    rescaledPixelSize: 50
             nSamples: []
             settings: [1x1 struct]
             roiStats: [1x1 struct]

>>

You can now overlay the bounding boxes with hBTview.view_acquire.overlayLastBoundingBoxes. You can even overlay the tile pattern:

z=hBT.recipe.tilePattern(false,false,hBT.autoROI.stats.roiStats.BoundingBoxDetails);
hBTview.view_acquire.overlayTileGridOnImage(z)