Point region shape n x

Welcome!

This GUI allows you to pick particles & make Topaz commands (watch Topaz presentation). General workflow:
→ → → →
Denoise
images		 Normalize &
downsample images		 Pick training
particles, or
analyze picks		 Train a
neural network		 Extract particle
coordinates

Note: To used pre-trained picking models, first Pre-process then Extract using a pre-trained model (don't Pick or Train).

Citations: Bepler, T, Morin, A, Rapp, M, Brasch, J, Shapiro, L, Noble, AJ, Berger, B (2019). Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs. Nat Meth

Bepler, T, Kelley, K, Noble, AJ, Berger, B (2020). Topaz-Denoise: general deep denoising models for cryoEM and cryoET. Nat Comm

   Topaz-Denoise

2D Denoising:

Denoise Micrographs

Create a command for denoising micrographs.

Parameters:            Must modify Might modify Rarely modify


Input micrographs
Output folder
Model(s)
GPU/CPU device
Format

Advanced options
Patch size
Patch padding


Command:

topaz denoise /path/to/input/images/*.mrc --model unet --device 0 --format mrc --patch-size 1536 --patch-padding 384 --normalize --output /path/to/output/folder/

Denoise Particle Stack

Create a command for denoising a particle or micrograph stack.

Parameters:            Must modify Might modify Rarely modify


Input stack
Output stack
Model(s)
GPU/CPU device
Format


Command:

topaz denoise /path/to/particle/stack.mrcs --model unet --device 0 --format mrc --stack --normalize --output /path/to/denoised/stack.mrcs

Train New Denoising Model

Create a command for training a Topaz-Denoise model.

Parameters:            Must modify Might modify Rarely modify


Input folder #1
Input folder #2
Output files
Model architecture
Training criteria
GPU/CPU device
Number of CPUs

Advanced options
Training method
Optimizer
Learning rate
Crop size
Batch size
Number of epochs


Command:

topaz denoise --dir-a /path/to/input/images/half1/ --dir-b /path/to/input/images/half2/ --method noise2noise --arch unet --optim adagrad --lr 0.001 --criteria L2 --crop 800 --batch-size 4 --num-epochs 100 --device 0 --num-workers 16 --save-prefix /path/to/output/folder/and/files



3D Denoising:

Denoise Tomogram or 3D Volume

Create a command for denoising tomograms or 3D volumes.

Parameters:            Must modify Might modify Rarely modify


Input volumes
Output folder
Model
Gaussian filter
GPU/CPU device

Advanced options
Patch size
Patch padding


Command:

topaz denoise3d /path/to/input/volumes/*.mrc --model unet-3d-10a --device -2 --gaussian 0 --patch-size 96 --patch-padding 48 --output /path/to/output/folder/

Train New 3D Denoising Model

Create a command for training a 3D Topaz-Denoise model.

Parameters:            Must modify Might modify Rarely modify


Input folder #1
Input folder #2
Output files
Training criteria
GPU/CPU device
Number of CPUs

Advanced options
Weight decay
Momentum
# of train points
# of test points
Optimizer
Learning rate
Crop size
Minibatch size
Number of epochs
Save interval


Command:

topaz denoise3d -a /path/to/input/volumes/half1/ -b /path/to/input/volumes/half2/ --optim adagrad --lr 0.001 --criteria L2 --crop 96 --batch-size 10 --weight_decay 0 --momentum 0.8 --N-train 1000 --N-test 200 --num-epochs 100 --save-interval 5 --device -2 --num-workers 1 --save-prefix /path/to/output/folder/and/files



      Topaz Image Pre-processing Command Generator

Create a command for pre-processing images (downsampling and normalization) for use in this Topaz GUI and in Topaz training.


Parameters:            Must modify Might modify Rarely modify


Input micrographs
Output folder
Number of CPUs
GPU/CPU device
Scale factor

Advanced options
Output format
Sample
Number of iterations
Alpha
Beta


Commands:

topaz preprocess /path/to/input/images/*.mrc --scale 4 --sample 1 --num-workers -1 --format mrc,png --device 0 --niters 100 --alpha 900 --beta 1 --verbose --destdir /path/to/output/preprocessed/images/



Note: Denoising micrographs before preprocessing and before binning will make particle picking easier. Training on denoised micrographs may not work any better than the originals. It is recommended that you denoise micrographs for manual picking/inspection by using the Denoise tab on the right.


Pick Annotations Image Annotations × ↑ ↓ + −
  • Home
  • Project
    • Load
    • Save
    • Settings
    • Add local images
    • Add images from paths
    • Add images from URL
    • Add urls or paths from text file
  • Particle Picks
    • Export Picks as csv (s)
    • Import Picks from csv
    • Preview Picks
    • Download as Image
  • View
    • Toggle image grid view
    • Toggle left sidebar
    • Toggle image filename list
    • Toggle attributes editor
    • Toggle annotation editor (Space)
    • Show/hide pick boundaries (b)
    • Show/hide pick labels (l)
    • Show/hide cursor coordinates (x)
  • Help
Open Project Save Project Update Project Settings Import Picks (as csv) Export Picks (as csv) Switch to Image Grid View Previous Image Toggle Image List Next Image Zoom In Zoom Out Select All Picks Copy Picks Paste Picks Paste Pick in Multiple Images Undo Picks Pasted in Multiple Images Delete Pick
Sorry, your browser does not support HTML5 Canvas functionality which is required for this application.
Group by 
Selected 0 of 0 images in current group, show

Settings

Project Name
Default Path
If all images in your project are saved in a single folder, set the default path to the location of this folder. The VIA application will load images from this folder by default. Note: a default path of "./" indicates that the folder containing the topaz.html application file also contains the images in this project. For example: /datasets/VOC2012/JPEGImages/ or C:\Documents\data\ (note the trailing / and \)
Search Path List
If you define multiple paths, all these folders will be searched to find images in this project. We do not recommend this approach as it is computationally expensive to search for images in multiple folders.
    Particle Pick Label
    By default, each pick in an image is labelled using the region-id. Here, you can select a more descriptive labelling of regions.
    Pick Color
    By default, each pick is drawn using a single color. Using this setting, you can assign a unique color to picks grouped according to an attribute such as Topaz score.
    Particle Pick Label Font
    Font size and font family for showing pick labels.
    Preload Buffer Size
    Images are preloaded in buffer to allow smoother navigation of next/prev images. A large buffer size may slow down the overall browser performance. To disable preloading, set buffer size to 0.
    On-image Annotation Editor
    When a single particle pick is selected, the on-image annotation editor is activated which the user may use to update annotations of this pick. By default, this on-image annotation editor is placed near the selected pick.

    File Not Found

    Filename:

    We recommend that you update the default path in project settings to the folder which contains this image.

    A temporary fix is to use browser's file selector to manually locate and add this file. We do not recommend this approach because it requires you to repeat this process every time your load this project in the VIA application.

    1. Denoise MRCs (recommended): Use Topaz-Denoise to denoise full-size raw micrographs.
      • Denoising using the pre-trained models works best on K2, K3, Falcon II, and Falcon III micrographs. You may wish to make your own denoising model if the pre-trained models are not optimal.
    2. Preprocess MRCs: Use Topaz to downsample, normalize, and convert denoised micrographs to PNG (for use in Topaz picking GUI) and raw micrographs to MRC (for use with Topaz training).
      • Your particle must have a diameter (longest dimension) after downsampling of: 70 pixels or less for resnet8; 30 pixels or less for conv31; 62 pixels or less for conv63; 126 pixels or less for conv127.
      • Topaz normalization uses image statistics for all input micrographs. A Gaussian mixture model is used to effectively make the ice region of each micrograph visible, even if there is gold in the image.
    3. Pick Particles: Add images then pick particles:
      • To add images:
        • Click Add Images or Add Image Paths under 'Project' to the left
        • Note: This Topaz GUI accepts PNG, JPEG, and BMP files.
      • To pick particles:
        • Left-click on the center of particles for several micrographs.
        • To zoom in/out, either press Ctrl and scroll your mouse wheel on an image or press + or - (equal = will reset the zoom).
        • To delete a misplaced point, click the point again and press d.
        • To adjust a point's location, click the point again and either use the arrow keys on your keyboard or click-and-drag to move it.
        • Note: Topaz works best if you identify 500+ particles representing all orientations of the particle and if you pick across several representative micrographs. More training particles and views means more accurate Topaz-trained picks.
        • Note: The radius of the displayed pick is 5 pixels in un-zoomed images.
    4. Train a model: Use Topaz to train a neural network model to pick your particles.
      • Training often works best when the training radius is 3 pixels or less.
      • Several parameters are optimizable by cross-validation.
    5. Export Particles: To export particles, click Particle Picks → Export Picks in the top menu. These picks may then be used in Topaz training.
    6. Analyze Topaz Picks: Load Topaz picks by clicking Particle Picks → Import Picks.
      • Press L on your keyboard to show the particle number, then the Up ↑ key will toggle the Topaz particle scores. Zoom-in on the image with + to inspect.

      Note: It is good practice to save your particles/project regularly. Do this by clicking Particle Picks → Export Picks or Project → Save in the top menu.

      Note: Parameters are categorized by those that need to be modified, might need to be modified, are optimizable, and are rarely modified. Training parameters that are optimizable may be optimized using cross-validation.

      If you wish to work outside this GUI, follow the GitHub tutorials.
    Loading ...
        
    Topaz Version 0.2.5VIA Version 2.0.7 (modified)

    Topaz is a pipeline for particle detection in cryo-electron microscopy images using convolutional neural networks trained from positive and unlabeled examples. The original research can be found here. Topaz is released under the GPLv3 license.

    Topaz GUI uses VGG Image Annotator (VIA), which is an image annotation tool that can be used to define regions in an image and create textual descriptions of those regions. VIA is an open source project developed at the Visual Geometry Group and released under the BSD-2 clause license.

    Here is a list of some salient features of the Topaz GUI version of VIA:

    • based solely on HTML, CSS, and Javascript
    • can be used off-line (full application in a single HTML file that is < 0.5MB)
    • only requires a modern web browser (tested on Chrome, Firefox, Edge, and Safari; Note: Some features look/work differently in different browsers; Chrome is recommended)
    • import/export particle picks in csv file format

    For more Topaz details, visit http://cb.csail.mit.edu/cb/topaz/.

    For more VIA details, visit http://www.robots.ox.ac.uk/~vgg/software/via/.

     

    VIA is Copyright © 2016-2019, Abhishek Dutta,Visual Geometry Group, Oxford University and VIA Contributors.

    Copyright (c) 2016-2019, Abhishek Dutta, Visual Geometry Group, Oxford University and VIA Contributors. All rights reserved.

    Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

    Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

          Pick or Analyze Particles

      Pick particles for training networks or analyze Topaz picks. Follow the steps below:

      Particle picking for training

      1) To start picking, select images or add images from paths.
      2) Pick 100-1,000 particles (more is better) representing all orientations.
          • Pick across several representative images.
          • Not all particles need to be picked from each image.
      3) Export your particle picks and proceed to Topaz training.

      • Save your particles and project regularly. You may load them later to continue.

      • For further help, see the Getting Started page.

      Analyzing Topaz picks

      • Import your Topaz particle picks as a CSV file and review the picks.

        ◦ Press L on your keyboard to show the particle number, then the Up ↑ key will toggle the Topaz particle scores. Zoom-in on the image with + to inspect.

      Or

      • Load your micrographs and particles in .star format into Relion. Picks may be colored based on Topaz score by using the AutopickFigureOfMerit column in the .star file.

        Topaz Training Command Generator

    Create a command for training a model using particle picks.


    Parameters:            Must modify Might modify Optimizable Rarely modify


    Training images folder
    Training particles
    Output
    Particle radius
    Autoencoder
    Number of epochs
    GPU/CPU device
    Number of CPUs
    CNN model
    Method
    Num particles/image
    K-fold

    Advanced options
    Fold
    Image extension
    Units
    Dropout
    Batch norm
    Unit scaling
    Scaled num of units
    L2
    Learning rate
    Minibatch size
    Minibatch balance
    Epoch size
    Test batch size


    Command:

    topaz train --train-images /path/to/preprocessed/images/ --train-targets /path/to/training_particles.csv --k-fold 5 --fold 0 --radius 3 --model resnet8 --image-ext .mrc --units 32 --dropout 0.0 --bn on --unit-scaling 2 --ngf 32 --method GE-binomial --autoencoder 0 --num-particles 300 --l2 0 --learning-rate 0.0002 --minibatch-size 256 --minibatch-balance 0.0625 --epoch-size 5000 --num-epochs 10 --num-workers -1 --test-batch-size 1 --device 0 --save-prefix /output/path/model --output /output/path/results.txt



       Topaz Picks Extraction Command Generator

    Create commands for extracting and converting particle coordinates.


    Parameters:            Must modify Might modify Rarely modify


    Input micrographs
    Model
    Output filenames
    Particle radius
    Upscale picks
    Number of CPUs
    GPU/CPU device

    Advanced options
    Threshold
    Batch size
    Minimum radius
    Maximum radius
    Step radius


    Commands:

    topaz extract /path/to/preprocessed/images/*.mrc --model resnet16_u64 --radius 8 --threshold -6 --up-scale 1 --batch-size 1 --min-radius 5 --max-radius 100 --step-radius 5 --num-workers -1 --device 0 --output /path/to/extracted/particles.txt

    topaz convert /path/to/extracted/particles.txt --verbose 1 --output /path/to/extracted/particles.star

    topaz convert /path/to/extracted/particles.txt --verbose 1 --output /path/to/extracted/particles.csv



       More Tools
    Convert | Threshold Picks

    Create a command for converting between Topaz GUI CSV files, STAR files, or Topaz txt files.

    Parameters:            Must modify Might modify Rarely modify


    Input file
    Output file
    Threshold
    Upscale picks
    Downscale picks

    Specific filetype options
    Voltage
    Detector pixelsize
    Magnification
    Amplitude contrast
    Box size
    Image extension


    Command:

    topaz convert /path/to/input/coordinate/file.[csv|star|txt] --threshold -9999 --up-scale 1 --down-scale 1 --voltage -1 --detector-pixel-size -1 --magnification -1 --amplitude-contrast -1 --boxsize 0 --image-ext .mrc --verbose 1 --output /path/to/output/coordinate/file.[csv|star|txt]

    Extract Particle Stack

    Create a command for extracting a particle stack from micrographs using a Topaz .txt coordinate file.

    Parameters:            Must modify Might modify


    Input picks
    Input micrographs
    Output stack
    Boxsize
    Threshold
    Image extension
    Resize particles


    Command:

    topaz particle_stack /path/to/Topaz/coordinate/file.txt --image-root /path/to/full/micrographs/ --size 192 --threshold -9999 --resize -1 --image-ext .mrc --output /path/to/output/particles.mrcs

    Topaz in RELION, CryoSPARC, Scipion, & Appion

    Topaz picking and denoising is integrated into RELION, CryoSPARC, Scipion, and Appion!

  1. RELION-Topaz tutorial is here: https://github.com/tbepler/topaz/tree/master/relion_run_topaz


  2. CryoSPARC-Topaz tutorial is here: https://guide.cryosparc.com/processing-data/all-job-types-in-cryosparc/deep-picking/deep-picking


  3. Scipion-Topaz information is here: https://github.com/scipion-em/scipion-em-topaz

    4. Topaz Workshop

    To request a Topaz Workshop for academic or non-academic purposes, send a request to <alexjnoble [at] gmail [dot] com> and <tbepler [at] gmail [dot] com>.