Interoperability & Matrix Converter

Pillow ↔ OpenCV ↔ NumPy ↔ PyTorch Matrix

Pick a source and a target format to get a complete, runnable conversion script, with the RGB/BGR channel order, float scaling and tensor layout handled. All 36 combinations are run with real Pillow, OpenCV, NumPy and PyTorch when the site is built, and each one is checked to print the same pixel colour.

1. Select Source & Destination Formats

Choose your input container and target pipeline:

Source Format (Input)

Target Format (Output)

Pillow (RGB) → OpenCV (BGR)

Requires channel swap via cv2.COLOR_RGB2BGR and buffer export with np.array().

2. Generated Production Code

Copy-paste ready function with optimal memory management:

🧠 Channel Layout & Memory Mechanics

Understanding color channel ordering and memory layouts across libraries prevents subtle bugs and maximizes inference frame rates:

Pillow & Web Standards (RGB)

Standard Red-Green-Blue order. Pixels are indexed as [R, G, B] in 8-bit unsigned integers (uint8, range 0–255).

OpenCV Legacy Architecture (BGR)

Uses Blue-Green-Red order due to early camera frame grabber hardware standards in the 1990s. Forgetting to swap channels causes blue-tinted faces and inverted colors.

📚 Production Conversion Recipes

Zero-copy conversions, web streaming, deep learning tensors, and video overlay pipelines:

1. Pillow Image to OpenCV Mat (cv2.Mat)

Computer Vision

Converts a PIL Image to a NumPy ndarray with channel order converted from RGB to BGR for OpenCV filtering, feature matching, or edge detection.

import cv2
import numpy as np
from PIL import Image

def pil_to_cv2(pil_img):
    """
    Converts a PIL Image (RGB) to an OpenCV BGR numpy array.
    """
    # 1. Convert to RGB mode if not already (handles RGBA, L, P)
    if pil_img.mode != "RGB":
        pil_img = pil_img.convert("RGB")
        
    # 2. Convert to numpy array
    rgb_arr = np.array(pil_img)
    
    # 3. Swap RGB channels to BGR
    bgr_arr = cv2.cvtColor(rgb_arr, cv2.COLOR_RGB2BGR)
    return bgr_arr

# Usage
pil_image = Image.open("sample.jpg")
cv2_mat = pil_to_cv2(pil_image)
# cv2.imshow("OpenCV Window", cv2_mat)

2. OpenCV Mat to Pillow Image

Core Pipeline

Converts an OpenCV BGR frame or image back into a PIL Image for drawing crisp anti-aliased typography, compositing, or saving as optimized JPEG/PNG/WebP.

import cv2
from PIL import Image

def cv2_to_pil(cv2_mat):
    """
    Converts an OpenCV BGR numpy array to a PIL Image (RGB).
    """
    # Swap BGR to RGB
    rgb_arr = cv2.cvtColor(cv2_mat, cv2.COLOR_BGR2RGB)
    
    # Instantiate PIL Image from memory buffer
    return Image.fromarray(rgb_arr)

# Usage
frame = cv2.imread("opencv_input.jpg")
pil_img = cv2_to_pil(frame)
# pil_img.save("converted_output.png")

3. Base64 Data URL to PIL Image & OpenCV (Web API Decoders)

REST APIs & Fullstack

Directly decodes incoming Base64 JSON strings from frontend canvas uploads without touching the hard drive.

import io
import base64
from PIL import Image

def base64_to_pil(base64_str: str) -> Image.Image:
    """
    Decodes a Base64 string (with or without 'data:image/...;base64,' prefix) to PIL Image.
    """
    if "," in base64_str:
        base64_str = base64_str.split(",", 1)[1]
        
    image_bytes = base64.b64decode(base64_str)
    return Image.open(io.BytesIO(image_bytes))

def pil_to_base64(pil_img: Image.Image, format="JPEG") -> str:
    """
    Encodes PIL Image to Base64 Data URI string.
    """
    buffer = io.BytesIO()
    pil_img.save(buffer, format=format)
    encoded = base64.b64encode(buffer.getvalue()).decode("utf-8")
    return f"data:image/{format.lower()};base64,{encoded}"

4. PyTorch Tensor (C, H, W) ⇄ Pillow Image (H, W, C)

Deep Learning & AI

Transposes channels between PyTorch format (Channels, Height, Width) normalized in [0.0, 1.0] and Pillow (Height, Width, Channels) in [0..255].

import torch
import numpy as np
from PIL import Image

def pil_to_tensor(pil_img: Image.Image) -> torch.Tensor:
    """
    Converts PIL Image to PyTorch FloatTensor (C, H, W) scaled [0.0, 1.0].
    """
    arr = np.array(pil_img.convert("RGB"), dtype=np.float32) / 255.0
    # Transpose from (H, W, C) to (C, H, W)
    tensor = torch.from_numpy(arr).permute(2, 0, 1)
    return tensor

def tensor_to_pil(tensor: torch.Tensor) -> Image.Image:
    """
    Converts PyTorch FloatTensor (C, H, W) in range [0, 1] to PIL Image (H, W, C).
    """
    # Squeeze batch dimension if present (1, C, H, W) -> (C, H, W)
    if tensor.dim() == 4:
        tensor = tensor.squeeze(0)
        
    arr = tensor.detach().cpu().permute(1, 2, 0).numpy()
    arr = (np.clip(arr, 0.0, 1.0) * 255.0).astype(np.uint8)
    return Image.fromarray(arr)

5. Real-Time Video Stream Unicode Text Overlay (OpenCV + PIL)

Webcam & Video Streams

OpenCV's cv2.putText() lacks Unicode and custom TTF support. This recipe blends Pillow typography onto live OpenCV camera feeds seamlessly.

import cv2
import numpy as np
from PIL import Image, ImageDraw, ImageFont

def draw_unicode_text_on_cv2(cv2_frame, text, pos=(50, 50), font_size=32, text_color=(0, 255, 0)):
    """
    Draws beautiful Unicode / TrueType text on an OpenCV video frame without jagged edges.
    """
    # 1. Convert BGR cv2 frame to RGB PIL Image
    pil_img = Image.fromarray(cv2.cvtColor(cv2_frame, cv2.COLOR_BGR2RGB))
    draw = ImageDraw.Draw(pil_img)
    
    # 2. Load TrueType font (or default)
    try:
        font = ImageFont.truetype("arial.ttf", font_size)
    except IOError:
        font = ImageFont.load_default(size=font_size)   # scalable built-in font (Pillow 10.1+)
        
    # 3. Draw text with antialiasing
    draw.text(pos, text, font=font, fill=text_color)
    
    # 4. Convert back to OpenCV BGR
    return cv2.cvtColor(np.array(pil_img), cv2.COLOR_RGB2BGR)

📊 Framework Data Structure Reference

Library / Framework	Channel Order	Tensor Shape	Data Type	Value Range
Pillow (PIL.Image)	RGB or RGBA	`(Width, Height)` tuple	C structure	0 to 255 (8-bit)
OpenCV (cv2.Mat)	BGR or BGRA	`(Height, Width, Channels)`	`np.uint8`	0 to 255
NumPy Array	RGB (by convention)	`(Height, Width, Channels)`	`np.uint8` or `float32`	0..255 or 0.0..1.0
PyTorch (torch.Tensor)	RGB	`(Batch, Channels, Height, Width)`	`torch.float32`	0.0 to 1.0

❓ Frequently Asked Questions

np.asarray() reuses the existing underlying memory buffer of the PIL image whenever possible (zero-copy), avoiding a costly heap allocation. np.array() always performs a full memory copy of the pixel buffer.

A single-channel grayscale image in OpenCV has shape (Height, Width). In Pillow, it corresponds to mode "L". You can directly call Image.fromarray(gray_cv2_mat) without needing color space conversion.

Pillow expects standard np.uint8 arrays with values in [0, 255]. If your NumPy array is float32 or float64 (e.g., after neural network inference), you must multiply by 255, clamp with np.clip(0, 255), and cast with .astype(np.uint8) before calling Image.fromarray().