PipeDream is an asynchronous pipeline parallel strategy for training large neural networks. It adds inter-batch pipelining to intra-batch parallelism to further improve parallel training throughput, helping to better overlap computation with communication and reduce the amount of communication when possible.

**Sample Consistency Network (SCNet)** is a method for instance segmentation which ensures the IoU distribution of the samples at training time are as close to that at inference time. To this end, only the outputs of the last box stage are used for mask predictions at both training and inference. The Figure shows the IoU distribution of the samples going to the mask branch at training time with/without sample consistency compared to that at inference time.

SCNet

SCNet: Training Inference Sample Consistency for Instance Segmentation

PipeDream

Branch attention can be seen as a dynamic branch selection mechanism: which to pay attention to, used with a multi-branch structure.

Year	2019
Data Source	CC BY-SA - https://paperswithcode.com