AI-Powered Tools for Complex Selections in Photoshop

We're going to start using AI-powered Select and Mask tools. We’ve already talked a bit about making selections, and now I’ll go a little deeper into how these tools work, especially for more complex situations.

I’ll start by opening this image. One thing you’ll notice right away is that we have quick options such as Select Subject and Remove Background. If I click Select Subject, Photoshop automatically selects the subject. From there, I can create a mask, which effectively removes the background. If I click Remove Background instead, Photoshop performs the same action in a single step. As you can see, both methods produce a mask.

I’m going to undo everything and start again. As mentioned in a previous video, I strongly recommend adjusting your preferences before working with these tools. Go to Photoshop > Settings > Image Processing. On Windows, this is located under Edit > Preferences. In the Image Processing section, you’ll see options for Select Subject and Remove Background. I recommend setting both to Cloud, as cloud processing generally produces better results than using the device alone. I’ve already covered this in another video, but it’s worth repeating here.

Next, let’s look at the selection tools. Photoshop offers tools like the Lasso and Selection Brush, but the one powered by AI is the Object Selection Tool. When you activate it, Photoshop can automatically detect people in the image. If you click Select People, Photoshop recognizes the person and allows you to choose specific features, such as facial elements or clothing. This level of control is extremely useful.

In addition to selecting people automatically, you can also manually select specific parts of an image using the same tool. Depending on whether the tool is set to Rectangle or Lasso, you can draw a selection around a specific area. For example, if I want to select just the hand, I can click and drag a rectangle around it, and Photoshop will refine the selection automatically.

If the selection includes areas you don’t want, you can hold down the Option key on Mac or Alt on Windows and subtract from the selection. This allows you to isolate a specific area, such as the hand, with greater precision. While these tools are powerful, this session is primarily focused on removing backgrounds, especially when dealing with more complex edges like hair.

Let’s look at a more challenging example. I’ll open another image in Photoshop. While Photoshop can recognize the entire person automatically, I’m going to switch to Select and Mask to show you how AI improves results in more complex cases. You can access this by going to Select > Select and Mask.

You don’t have to use Select and Mask in every case, but it offers several AI-assisted tools that make it especially useful for difficult selections. Once inside Select and Mask, the first thing to review is the View settings in the top-right corner. Here, you can adjust the transparency using the slider. I often choose Overlay as the view mode because it lets me clearly see what’s selected. You can also change the overlay color, for example, switching it to green, and increasing the opacity to make the selection easier to evaluate.

Once the view is set, I click Select Subject. Because we’ve already set the preference to Cloud, Photoshop automatically processes the selection using Adobe’s servers. This generally produces a cleaner and more accurate result. I can further increase the overlay opacity and zoom in to inspect the selection.

At this stage, the selection looks quite good, but there are still some areas that need refinement. I can use the available tools to improve those edges. For example, clicking Refine Hair applies AI refinement across the entire hair area with a single click. If there are still issues, I can adjust Shift Edge slightly to the left, which often improves edge quality.

It’s important to note that the main AI processing happens during the selection itself—especially when using cloud processing. The sliders and fine-tuning tools are used to polish the result after the AI has done most of the work.

Once I’m satisfied, I choose Output to Layer Mask and click OK. This creates a clean mask. I can zoom out and place another image behind the subject to test the result. When viewed up close, the selection looks quite solid. While further refinement is possible using advanced techniques, that’s covered in a separate, more advanced Photoshop class. The goal here is to show that this workflow is heavily powered by AI.

Now, let’s look at another example where the tool struggles a bit. I’ll open an image of an eagle. This seems like an easy case because there’s a strong contrast between the white feathers and the green background. I’ll go to Select > Select and Mask and click Select Subject. Photoshop does a good job of recognizing the bird.

However, if I want to place the eagle on a white background, issues start to appear. Switching the view to On White and increasing opacity reveals green fringing around the edges. Adjusting Shift Edge helps a bit, but it’s not perfect. Clicking Refine Hair doesn’t help much in this case, since this isn’t hair.

I can experiment with additional options, such as adding a slight feather or enabling Decontaminate Colors. Decontaminate Colors can sometimes improve edge contamination, but it’s not always ideal. When I apply it and click OK, Photoshop creates a new layer with a modified mask.

If I disable the mask temporarily by holding Shift and clicking on it, you can see what Photoshop is doing behind the scenes. It’s painting the edge pixels white to counteract the green contamination. While this can help in some cases, it’s not always visually clean.

There are more advanced techniques to solve these issues, such as working on separate layers, changing blending modes, or manually refining edges. The key point is that AI is powerful but not perfect. Results depend heavily on the image itself, including background color, contrast, and edge complexity.

The best approach is to test these tools on different images and understand their strengths and limitations. In the next video, we’ll continue exploring additional AI-powered tools in Photoshop.