GETTING STARTED WITH MIDJOURNEY’S IMAGE TO VIDEO GENERATION

Midjourney Ai Art Design Course. A quick 5 minute read.

Tags: Image to Video, I2V, Midjourney v7

INTRO

Text-to-Image generation is Midjourney’s bread and butter, and you’re all comfortably familiar with how this all works. Just type in your Prompt and out pops your image “like magic”. Image-to-Video (I2V) generation, however, is a whole different level of bedeviled sorcery. Midjourney now capably joins the I2V heavy-hitters like Kling, Runway and Soma… and like its competitors, Midjourney is not without its learning curve or its quirks.

PART 1. WHAT TO EXPECT WITH MIDJOURNEY IMAGE-TO-VIDEO

If you’re familiar with Midjourney, especially the standalone app, video generation will be a smooth transition. Unlike Text to Image, Image to Video needs… well, an Image to start with. In classic Midjourney fashion, they’ll be generated in sets of 4, but you can change the Batch Size to 2 or 1. You can also set an end frame for Midjourney to transition to or even set it to Loop (perfect for Social Media).

The biggest parameter to consider is High versus Low Mode – the binary methods Midjourney uses to tweak Video generation. Low Mode is used for subtle animations and ambience – or for very focused prompts on simple actions. High Mode is good for large transitions, big camera changes, or dynamic action shots. It may take awhile to intuit how these work with an infinite variety of subjects (or don’t work as advertised at all!). But just between us, you should always generate in High and Low Mode to get the greatest number of surprises.

When Midjourney is done generating, you can mouse over the entire row to see a preview of all the videos. Conveniently, Midjourney will play these videos in a loop so you can test if your purposefully looping video is as seamless as you need it to be. There are innumerable ways to save when you right-click; from Raw mp4s to gifs to files perfect for Social Media as-is. 

FROM MIDJOURNEY’S OFFICIAL TUTORIAL

Midjourney Image to Video has some light tutorials on the official site. For your convenience (and to constantly prove I ain’t making anything up), here are a few key tips from the official Midjourney documentation.

• All plan tiers can generate videos in Fast Mode, but only Pro and Mega plans can generate videos in Relax Mode (SD resolution only).
• Standard, Pro, and Mega plans can generate HD resolution videos, but only in Fast Mode.
• Videos cost more GPU time compared to regular images. For a detailed breakdown of costs, see the Batch Size section below.
• Video generations are only compatible with video-specific parameters: –motion low, –motion high, –raw, –loop, –end, and –bs #.
• The Animate Manually link will give you an opportunity to adjust the text prompt in the Imagine bar before generating the video.
• The Auto buttons will automatically start generating a video using your image.
• The Loop buttons will automatically start generating a looping video using your image (meaning the starting frame and ending frame are the same image).
• For information on “Low Motion” and “High Motion” check out the Video Motion section below.
• Note: Any image parameters used to generate your original image will be automatically removed when generating videos.

.

WANT YOUR OWN 1-ON-1 MENTORSHIP IN MIDJOURNEY?

TRUSTED LEADERSHIP . REAL EDUCATION . PERSONALIZED CLASSES . CERTIFICATION . EMPATHY

Don’t let the future terrify you. Create your better tomorrow, today. 

Join my exclusive one-on-one Mentorship program in Midjourney for only $3997 (payments as low as $200/mo). 

Seating is extremely limited to only 5 at a time.

Discover your future and register for free

.

PART 2. HOW TO GET THE BEST RESULTS WITH MIDJOURNEY IMAGE TO VIDEO

Better you hear this from old Uncle Johnny: you need to accept that all Image to Video generation is a roulette wheel spin. Video has a spectacular randomness that far exceeds the relative stability of Text to Image. So the strategy is simple: be far more granular with Prompting, or far more accepting of the chaos.

Try to use Low Mode for subtle motion or prompts that focus on one simple, singular concept. Since the on-screen action should be appreciably downplayed, Prompt feelings and moods as well as actions. Use High Mode for action shots and dynamic camera changes. I believe High Mode also tries a bit harder to interpret things it can’t see: character’s clothing behind their back, offscreen scenery, etc. As so often mentioned, always try both High and Low with every generation.

But mostly, the best advice is to keep your expectations low and your mind open. Video generation is less about capturing the painstaking Kubrick-esque choreography in your head and much more about finding something that makes you snap your fingers and exclaim, “That’s it, that’s the one!”. Budget your time, and work in 30 minute to hour long blocks to keep yourself from dumping all your fast hours or just mindlessly generating without refinement. Don’t stay focused, stay receptive. 

A NOTE ABOUT EXTENDING VIDEO CLIPS

There is an important feature of Midjourney’s Image to Video I haven’t discussed, largely because I think it’s actually an advanced use-case. Defaultly, Midjourney makes your videos in 5 second clips, but you can extend them. Depending on your needs, this may be a much bigger feature for you than the layperson:

• Video generations start at 5 seconds, but it doesn’t have to stop there. After your video is created, you’ll see options to extend it further. This will add another 4 seconds to your video.
• Feel free to extend your video up to 4 times, until it reaches 21 seconds, which is the maximum length available.
• Look for the “Extend Auto” and “Extend Manual” buttons when you hover over your video on the Create page, and below the Creation Actions section when you open your video.
• Extend Auto: This button extends your video automatically, using the original prompt.
• Extend Manual: Use the manual option if you’d like to change your prompt text before extending the video. This way, you can tweak or add new elements before submitting for the longer video.

.

PART 3. THE DANGERS OF MIDJOURNEY IMAGE TO VIDEO.

Watch out, Midjourney now officially comes with a warning label from me! And I’m only half-joking here, you need to watch your ass around Midjourney’s Image-to-Video. No, it’s not a security risk and it won’t press a knife to your throat (it will go for your wallet, though!). Be careful around Midjourney Image to Video… for reasons that are actually quite unusual.

Midjourney I2V will burn through your Fast Hours, and it only uses fast hours (unless you have the relatively expensive Pro subscription). A good Rule of Thumb is that 1 hour of real-world Video work tears through about 3 Fast Hours (!). At the default 15 Hours for the Standard subscription, that roughly translates to 3 hours of work per month (!!). If you don’t need the quartet spread of options, definitely drop your Batch Size down to 2 or even 1. Consider only working for a set period of time to control your time spending. I can not stress this enough: you may go through your month of Fast Hours in a matter of a few days!

Which is also a problem because I2V is extremely addictive, like Vegas addictive. Just one more pull can lead you down a new path, or put your last favorite generation to shame. Midjourney is quite fast these days, so I’d bank your entire reserve of Fast Hours exclusively for Video. When you work, put twice the effort into Prompting and remember that for best results you’ll be using both Fast and Low mode – which means you need to calculate double the hours per Prompt!

MIDJOURNEY VIDEO RESOLUTION

Midjourney’s resolution on videos has a little bit of nuance to it. For most of you, this won’t matter much, as those of you with high resolution requirements would have a Pro subscription or even a homebrew Stable Diffusion solution by now. But just in case you’re curious, here’s the definitive answer to Midjourney’s video resolution questions.

• Midjourney videos are generated in 480p quality, also called Standard Definition (SD). If you have the Standard, Pro, or Mega plan you can also generate videos in 720p quality, or High Definition (HD), by adjusting your Video Resolution under More Options in the settings panel of the Imagine bar.
• The exact shape and size of your video is based on the aspect ratio of your starting image. If you upload your generated video to another video-sharing platform, the quality will be classified as 480p or 720p.
• The table below shows how different starting image aspect ratios will affect your video’s pixel dimensions:

.

LET’S SUM UP

Midjourney’s Image to Video generation is a top-tier tool for movies and clips – just not for major motion pictures. Nonetheless, it’s streamlined process and zippy results make it’s a supremely powerful addition to the Midjourney suite of tools. But it’s not without it’s unique quirks.

• High Mode and Low Mode have their uses, but ultimately, so much is left to random happenstance that their impact is difficult to measure. You’re better off using both for every generation.
• Its focus is more bite-size focused for now: 5 second clips with extensions, easy Social Media saves and rapid generation.
• Midjourney Video will tear through your fast hours with an alarming burn rate. Bank all your hours and take frequent breaks.

Related Links and Further Reading

• Midjourney Image to Video Official Documentation
• The Number 1 Guide to Getting Started With Midjourney
• How to Make an Ai Portfolio

.

MIDJOURNEY Ai LONGTAIL SEO SEARCH TERMS 

This may seem tangentially related, but many of my Midjourney Mentees are also web designers and have a passing knowledge of SEO. As one of the very few Ai practitioners with his own long running site, search engine optimization for terms is at the heart of my business. Many of you won’t care, but some of you always ask (and it’s very obscure knowledge only a few of

• how to get started with Midjourney video
• Midjourney image to video tutorial for beginners
• beginner’s guide to Midjourney image to video
• turning still images into video with Midjourney
• how to animate Midjourney images
• Midjourney video generation explained
• getting started with Midjourney’s new video feature
• how to use Midjourney’s image to motion tool
• step-by-step guide Midjourney video
• Midjourney video commands for beginners
• Midjourney image to video prompt guide
• Midjourney video generation from a single image
• Midjourney animate a photo tutorial
• turning a Midjourney still into a video clip
• how to create video with Midjourney
• Midjourney image to video settings
• beginner Midjourney video guide for artists
• how to make animated videos with Midjourney
• best practices for Midjourney image to video
• Midjourney video for social media content
• creating a short video from a Midjourney image
• how to get smooth motion in Midjourney videos
• beginner’s guide to Midjourney motion
• Midjourney video tutorial for creators
• using Midjourney to create video art
• Midjourney video generation walkthrough
• turning an image into an MP4 with Midjourney
• how to add motion to a Midjourney image
• Midjourney video generation from custom image
• Midjourney video for beginners guide