我们在做RTSP|RTMP播放的时候,遇到好多开发者,他们的视觉算法大多运行在python下,需要高效率的实现C#和Python的视频数据交互,常用的方法如下:
方法一:通过HTTP请求传输视频数据
服务器端(Python)
使用Flask或Django等Web框架创建一个HTTP服务器,通过API接口提供视频数据。
from flask import Flask, send_file, request, jsonify
import cv2
import io
import base64 app = Flask(__name__) def get_video_frame(): # 读取视频帧 cap = cv2.VideoCapture('your_video.mp4') ret, frame = cap.read() cap.release() if not ret: return None, "No more frames" # 转换为RGB格式 frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) # 转换为字节流 ret, buffer = cv2.imencode('.jpg', frame_rgb) if not ret: return None, "Could not encode frame" # 将字节流转换为base64字符串 img_str = base64.b64encode(buffer).decode('utf-8') return img_str, 200 @app.route('/video_frame', methods=['GET'])
def video_frame(): img_str, status_code = get_video_frame() if img_str is None: return jsonify({"error": status_code}), status_code return jsonify({"image": img_str}) if __name__ == '__main__': app.run(debug=True)
客户端(C#)
使用HttpClient从Python服务器获取视频帧,并将其显示在PictureBox中。
using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Net.Http;
using System.Windows.Forms; public class VideoForm : Form
{ private PictureBox pictureBox; private HttpClient httpClient; public VideoForm() { pictureBox = new PictureBox { Dock = DockStyle.Fill, SizeMode = PictureBoxSizeMode.StretchImage }; this.Controls.Add(pictureBox); httpClient = new HttpClient(); Timer timer = new Timer(); timer.Interval = 100; // Adjust the interval as needed timer.Tick += Timer_Tick; timer.Start(); } private async void Timer_Tick(object sender, EventArgs e) { try { HttpResponseMessage response = await httpClient.GetAsync("http://localhost:5000/video_frame"); response.EnsureSuccessStatusCode(); string jsonResponse = await response.Content.ReadAsStringAsync(); dynamic jsonData = Newtonsoft.Json.JsonConvert.DeserializeObject(jsonResponse); string base64String = jsonData.image; byte[] imageBytes = Convert.FromBase64String(base64String); using (MemoryStream ms = new MemoryStream(imageBytes)) { Image image = Image.FromStream(ms); pictureBox.Image = image; } } catch (Exception ex) { MessageBox.Show($"Error: {ex.Message}"); } } [STAThread] public static void Main() { Application.EnableVisualStyles(); Application.Run(new VideoForm()); }
}
方法二:通过ZeroMQ传输视频数据
ZeroMQ是一种高性能的异步消息库,适用于需要低延迟和高吞吐量的应用。
服务器端(Python)
使用pyzmq
库发送视频帧。
import zmq
import cv2
import numpy as np context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:5555") cap = cv2.VideoCapture('your_video.mp4') while True: ret, frame = cap.read() if not ret: break frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) frame_bytes = frame_rgb.tobytes() socket.send_string("FRAME", zmq.SNDMORE) socket.send(frame_bytes, flags=0, copy=False)
客户端(C#)
使用NetMQ
库接收视频帧。
using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
using System.Threading.Tasks;
using System.Windows.Forms;
using NetMQ;
using NetMQ.Sockets; public class VideoForm : Form
{ private PictureBox pictureBox; private SubscriberSocket subscriber; public VideoForm() { pictureBox = new PictureBox { Dock = DockStyle.Fill, SizeMode = PictureBoxSizeMode.StretchImage }; this.Controls.Add(pictureBox); var address = "tcp://localhost:5555"; using (var context = NetMQContext.Create()) { subscriber = context.CreateSubscriberSocket(); subscriber.Connect(address); subscriber.Subscribe("FRAME"); Task.Run(() => ReceiveFramesAsync(subscriber)); } } private async Task ReceiveFramesAsync(SubscriberSocket subscriber) { while (true) { var topic = await subscriber.ReceiveFrameStringAsync(); var frameBytes = await subscriber.ReceiveFrameBytesAsync(); if (topic == "FRAME") { int width = 640; // Adjust based on your video resolution int height = 480; // Adjust based on your video resolution int stride = width * 3; Bitmap bitmap = new Bitmap(width, height, PixelFormat.Format24bppRgb); BitmapData bitmapData = bitmap.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.WriteOnly, bitmap.PixelFormat); Marshal.Copy(frameBytes, 0, bitmapData.Scan0, frameBytes.Length); bitmap.UnlockBits(bitmapData); pictureBox.Image = bitmap; } } } [STAThread] public static void Main() { Application.EnableVisualStyles(); Application.Run(new VideoForm()); }
}
方法三:通过共享内存或文件
如果C#和Python运行在同一台机器上,可以通过共享内存或文件系统进行数据交换。这种方法相对简单,但性能可能不如前两种方法。