We propose a framework for rotation and translation covariant deep learning using SE(2) group convolutions, cf.[1]. The group product of the special Euclidean motion group SE(2) describes how a concatenation of two roto-translations results in a net roto-translation. We encode this geometric structure into convolutional neural networks (CNNs) via SE(2) group convolutional layers, which fit into the standard 2D CNN framework, and which allow to generically deal with rotated input samples without the need for data augmentation. We introduce three layers: a lifting layer which lifts a 2D image to an SE(2)-image, i.e., 3D data whose domain is SE(2); a group convolution layer from and to an SE(2)-image; and a projection layer from an SE(2)-image to a 2D image. The lifting and group convolution layers are SE(2)-covariant (i.e. the output roto-translates with the input). The projection layer, a maximum intensity projection over rotations, makes the full CNN rotation invariant. A typical SE(2)-CNN consist of a rotationally-invariant feature encoding part (a sequence of a lifting layer, group convolution layers and a projection layer) followed by (fully connected) output layers in which the outputs are mapped to class probabilities via a sigmoid activation function, a scalar bias and weights. In this work we consider two class problems where we use cross-entropy as a loss function. We address three different medical imaging problems: (1) Mitosis detection in histopathology, (2) Vessel segmentation in retinal imaging, (3) Cell boundary detection in electron microscopy images. Each time we achieve state-of-the-art performance with our SE(2)-CNNs, without the need for data augmentation by rotation and with an increased performance compared to standard CNNs that do rely on augmentation. We show the advantage of including multiple orientation and group convolutions in a fair comparison where the total network capacity is kept constant, and where training and testing data is maintained.