Totally Hacky Way to Stream IP Camera in Colab - RTSP

UPDATED POST in this link: most free RTSP feeds are not working anymore; see the update for free and freemium alternatives for video feed ingestion testing.

In our previous publication, we leveraged the use of javascript into Colab to obtain a feed from a webcam local to your computer. Unfortunately, our knowledge of javascript is deficient; we really do not work with it, so when trying to obtain the same results for a streaming IP camera delivering a Real-Time Streaming Protocol (RTSP) video feed, we found ourselves placing the worst hack possible into action. First of all, at the point of writing this publication, we do not know how to modify appropriately and effectively the javascript generated webcam stream; on the other hand, we do know how to alter that stream to overlay detection information, so instead of overlaying detection information into the webcam stream we are going to overlay RSTP feeds from external cameras.

We are not repeating the code from our previous publication. This new overlaying function will add external RSTP feeds to our webcam input, completely masking it under an alpha channel with a value of 1. Again, we modify the image's resolution, just in case we hit video streams at high resolutions.

The overlay function that covers our webcam feed entirely and replaces it with the streaming video from the external source is this:

def overlay_rtsp(image, output_image):  

  stream = 'rtsp://demo:demo@ipvmdemo.dyndns.org:5541/onvif-media/media.amp?profile=profile_1_h264&sessiontimeout=60&streamtype=unicast'

  cap = cv2.VideoCapture(stream)
  ret, frame = cap.read()
  size = (600, 800)

  frame = cv2.resize(frame, size)  
  output_image[:, :, 0:3] = frame  
  output_image[:, :, 3] = 1

  # Add our logo if present:
  try:
    logo_file = '/content/ostirion_logo.jpg'
    img = cv2.imread(logo_file)
    new_size = (100, 100)
    img = cv2.resize(img, new_size, interpolation = cv2.INTER_AREA)
    lim = -new_size[0]-1
    output_image[lim:-1, lim:-1, 0:3] = img
    output_image[lim:-1, lim:-1, 3] = 1
  except:
    pass

  return output_image

We can also modify the drawing array function to accept different video stream sources, overlaying RTSP feeds by default:

def get_drawing_array(image_array, video_width=800, video_height=600,
                      overlay_function=overlay_rtsp):    
    drawing_array = np.zeros([video_width, video_height, 4], dtype=np.uint8)
    drawing_array = overlay_function(image_array, drawing_array)
    drawing_array[:, :, 3] = (drawing_array.max(axis=2) > 0 ).astype(int)*255
    return drawing_array

The main streaming function can be launched, and the test RTSP should appear:

start_input()
label_html = 'Capturing IP Camera Stream.'
img_data = ''

while True:
    js_reply = take_photo(label_html, img_data)    
    if not js_reply:
        break

    image = js_reply_to_image(js_reply)
    drawing_array = get_drawing_array(image, overlay_function=overlay_rtsp) 
    drawing_bytes = drawing_array_to_bytes(drawing_array)
    img_data = drawing_bytes

This test RTSP feed is taken from this external source and may stop working; if you have your own internet-facing camera, you can change the URL in the overlay RTSP function to use your own. The rendering process is slow; it takes a few seconds to get a new frame, even if the source feed is at 30 or 60 FPS. However, it is sufficient for demonstration. You can open the test feed using VLC or similar video software to check the stream rate without being processed by the Colab Notebook.

This should be the notebook output video:

Do not hesitate to contact us if you require quantitative model development, deployment, verification, or validation. We will also be glad to help you with your machine learning or artificial intelligence challenges when applied to asset management, automation, or intelligence gathering from satellite, drone, or fixed-point imagery.

The notebook for this demonstration is in this link.