How to override NSTextView dragging behaviour without overriding mouseDown:?

I have an NSTextView subclass that overrides mouseDown: to allow for image resizing. If a user clicks and drags on the edges of an image, I implement custom behaviour that resizes the image (and shows resizing cursors). If the user clicks anywhere else, super's implementation is called. This all works great.

As of macOS 27, however, the transition to gesture recognisers instead of overriding mouseDown: means that I should probably be moving away from the above approach. NSTextView now uses the new NSTextSelectionManager to implement selection and dragging via gesture recognisers, although, according to the release notes:

Existing NSTextView subclasses that override mouseDown: continue to work through a binary-compatible fallback path. (163365571)

It's unclear whether this means that we therefore should still override mouseDown: for custom behaviour in NSTextView, but to me, this, along with the content of Tech Note TN3212, strongly implies that, although it will continue to work thanks to "a binary-compatible fallback", we should entirely move away from overriding mouseDown: in the future.

If that is indeed the case, how do we implement custom dragging behaviour--such as for resizing images as in my example--in NSTextView? There still seems to be no way of doing it other than overriding mouseDown:.

I had thought that I might be able to add an NSPanGestureRecognizer to the text view and have it fail via its delegate methods if the clicks were outside of an image's edges, but a pan gesture recogniser added to an NSTextView is entirely ignored, presumably because of the private gestures already added.

Fortunately everything continues to work for now, but I would like to update my code as much as possible.

Answered by Frameworks Engineer in 894667022

I'm really glad to hear that you're working on adopting gesture recognizers and exploring NSTextSelectionManager.

Quick clarification that I think matters here. On macOS 27 the selection interactions aren't handled by NSTextView's own event methods anymore. They're owned by NSTextSelectionManager, which installs and manages its own gesture recognizers on the text view. You can get to the manager through the new NSView.textSelectionManager property. Your pan recognizer got ignored because those selection recognizers begin right away and nothing told them to wait for yours, so yours never had a chance to start.

mouseDown: is still supported, not deprecated. What it does under the hood is worth knowing though. NSTextView sees the override and disables the NSTextSelectionManager gesture path for the whole view, falling back to the old NSEvent selection code. That's why your call-super path still behaves the way it always did. You also still need the override if you're deploying to anything older than 27, since the gesture path isn't there. The annoying part is that currently there isn't a great way to keep the override only for the older releases and still get the gesture-based selection on 27, because overriding the method is all-or-nothing per view. The best way to do this currently is providing different subclasses depending on what OS your running on, but ugh. We're still working on a better answer though so stay tuned.

How you move to gesture recognizers depends on how the image is in the view today.

If you're already vending the image through NSTextAttachmentViewProvider, you're in good shape. The attachment has its own view, so just add your resize recognizer to that view. A recognizer on a subview takes precedence over selection on its own, and selection everywhere else keeps working.

If you're drawing the image yourself or using an attachment cell, the cleanest move is to adopt NSTextAttachmentViewProvider so the image gets its own view, then do the above. That's what I'd recommend if it's at all practical for you.

If moving the image into its own view isn't an option and you need to keep drawing into the text view, you can add your own recognizer in your subclass. IMO a press gesture matches "drag on an edge" better than a pan here but you'll need to set up failure relationship yourself so selection waits on it. NSTextSelectionManager hands you its recognizers for this:

for (NSGestureRecognizer *selectionGesture in self.textSelectionManager.gesturesForFailureRequirements) {
    [selectionGesture requireGestureRecognizerToFail:myEdgeResizeGesture];
}

Then have myEdgeResizeGesture begin only when the press is on an image edge (gate it in gestureRecognizerShouldBegin: or the delegate), so it fails everywhere else and selection runs as normal. Skip the failure relationship and the selection recognizers just win, which is the behavior you ran into.

Accepted Answer

I'm really glad to hear that you're working on adopting gesture recognizers and exploring NSTextSelectionManager.

Quick clarification that I think matters here. On macOS 27 the selection interactions aren't handled by NSTextView's own event methods anymore. They're owned by NSTextSelectionManager, which installs and manages its own gesture recognizers on the text view. You can get to the manager through the new NSView.textSelectionManager property. Your pan recognizer got ignored because those selection recognizers begin right away and nothing told them to wait for yours, so yours never had a chance to start.

mouseDown: is still supported, not deprecated. What it does under the hood is worth knowing though. NSTextView sees the override and disables the NSTextSelectionManager gesture path for the whole view, falling back to the old NSEvent selection code. That's why your call-super path still behaves the way it always did. You also still need the override if you're deploying to anything older than 27, since the gesture path isn't there. The annoying part is that currently there isn't a great way to keep the override only for the older releases and still get the gesture-based selection on 27, because overriding the method is all-or-nothing per view. The best way to do this currently is providing different subclasses depending on what OS your running on, but ugh. We're still working on a better answer though so stay tuned.

How you move to gesture recognizers depends on how the image is in the view today.

If you're already vending the image through NSTextAttachmentViewProvider, you're in good shape. The attachment has its own view, so just add your resize recognizer to that view. A recognizer on a subview takes precedence over selection on its own, and selection everywhere else keeps working.

If you're drawing the image yourself or using an attachment cell, the cleanest move is to adopt NSTextAttachmentViewProvider so the image gets its own view, then do the above. That's what I'd recommend if it's at all practical for you.

If moving the image into its own view isn't an option and you need to keep drawing into the text view, you can add your own recognizer in your subclass. IMO a press gesture matches "drag on an edge" better than a pan here but you'll need to set up failure relationship yourself so selection waits on it. NSTextSelectionManager hands you its recognizers for this:

for (NSGestureRecognizer *selectionGesture in self.textSelectionManager.gesturesForFailureRequirements) {
    [selectionGesture requireGestureRecognizerToFail:myEdgeResizeGesture];
}

Then have myEdgeResizeGesture begin only when the press is on an image edge (gate it in gestureRecognizerShouldBegin: or the delegate), so it fails everywhere else and selection runs as normal. Skip the failure relationship and the selection recognizers just win, which is the behavior you ran into.

Thank you for such a fantastic and thorough answer! That covers pretty much everything I was missing and I really appreciate it.

You can get to the manager through the new NSView.textSelectionManager property.

I had been looking for this but in the wrong place - I was looking in NSTextView and NSTextInputClient and hadn't thought to look on NSView. That's great that this exists.

If you're already vending the image through NSTextAttachmentViewProvider, you're in good shape.

Unfortunately I'm unable to use NSTextAttachmentViewProvider, as I believe it is TextKit 2 only, whereas for now I need to use TextKit 1 until TK2 catches up a bit more (multiple text containers, table support and so on).

(Hmm, actually, is NSTextSelectionManager even used in TextKit 1? I notice its data source methods all use TK2 ranges and locations.)

After a quick test in a sample project, though, it looks like your suggestion of using textSelectionManager.gesturesForFailureRequirements should work perfectly. It seems that requireGestureRecognizerToFail: is only available in UIKit, not AppKit, but I was able to achieve the same effect using:

- (BOOL)gestureRecognizer:(NSGestureRecognizer *)gestureRecognizer shouldBeRequiredToFailByGestureRecognizer:(NSGestureRecognizer *)otherGestureRecognizer
{
    return [self.textSelectionManager.gesturesForFailureRequirements containsObject:otherGestureRecognizer];
}

One question regarding this though: a class dump reveals that NSTextView already implements several NSGestureRecognizerDelegate methods, which presumably means that if I implement the delegate methods myself in my subclass, I risk breaking standard behaviour. So am I right in thinking that I should avoid making my NSTextView subclass the delegate of my own gesture recogniser?

(Is there a reason NSTextView doesn't publicly declare the NSGestureRecognizerDelegate methods it conforms to so that we can override them? I notice it's the same with other views migrating to gesture recognisers, such as NSTableView. Sorry, that was two questions.)

The best way to do this currently is providing different subclasses depending on what OS your running on, but ugh. We're still working on a better answer though so stay tuned.

This is all really useful information, thanks. Given that I create my text views programmatically, I could use different subclasses, but I'll probably just get gesture recognisers working (so that I'm ready for the transition), and then stick to using mouseDown: for the time being, for backward compatibility.

Thanks again!

I had been looking for this but in the wrong place - I was looking in NSTextView and NSTextInputClient and hadn't thought to look on NSView. That's great that this exists.

The intent of NSTextSelectionManager is that it can be installed in any view to add text selection behaviors to any view and not just NSTextView subclasses.

Unfortunately I'm unable to use NSTextAttachmentViewProvider, as I believe it is TextKit 2 only.

Yep, that's unfortunate. Sigh

(Hmm, actually, is NSTextSelectionManager even used in TextKit 1? I notice its data source methods all use TK2 ranges and locations.)

NSTextSelectionManager uses the NSTextSelectionDataSource protocol to get the information it needs to communicate selection with the delegate. While these are using TK2 values we were able to have NSLayoutManager also conform to the datasource protocol so it can interact with NSTextSelectionManager.

It seems that requireGestureRecognizerToFail: is only available in UIKit, not AppKit

Whoops, looks like I need to update the header comment.

One question regarding this though: a class dump reveals that NSTextView already implements several NSGestureRecognizerDelegate methods, which presumably means that if I implement the delegate methods myself in my subclass, I risk breaking standard behaviour. So am I right in thinking that I should avoid making my NSTextView subclass the delegate of my own gesture recogniser?

The gesture delegate implementation in NSTextView and other classes is not public and we don't want clients to have to worry about overriding and possibly breaking our implementations. In Seed 1 NSTextView doesn't use the public NSGestureRecognizerDelegate protocol so you're safe to implement your own gesture recognizers there. We're looking at making additional changes throughout AppKit to ensure that you are able to safely subclass and implement gesture recognizers without future grief for either of us.

(Is there a reason NSTextView doesn't publicly declare the NSGestureRecognizerDelegate methods it conforms to so that we can override them? I notice it's the same with other views migrating to gesture recognisers, such as NSTableView. Sorry, that was two questions.)

With mouse events it is normal to override and just call super (or not) to get the standard behavior. All of the logic for handling the mouse event is generally in that one mouseDown implementation. With gesture recognizers the implementation is spread across many different optional methods with potentially subtle timing and behavioral changes over time that would be hard to maintain (see the other discussion about when exactly the selection changes when a context menu is presented). With GRs you don't want to override the implementations but instead you set up the failure relationships between the gestures and you are then isolated from the implementation of other gestures in the system.

NSTextSelectionManager uses the NSTextSelectionDataSource protocol to get the information it needs to communicate selection with the delegate. While these are using TK2 values we were able to have NSLayoutManager also conform to the datasource protocol so it can interact with NSTextSelectionManager.

Ah, that's great, thank you. I really appreciate the work still going in to keep TextKit 1 up to date with things like this. I've built a lot of TK2 alternatives to my TK1 code ready to switch one day, and I use TK2 where I can, but for now TK1 remains the only option for some things. (Not that I'm in any rush for TK2 to catch up, as I do not look forward to the day I have to rewrite in TK2 my code that handles laying out multiple pages with footnotes, widow and orphan control and so on.)

In Seed 1 NSTextView doesn't use the public NSGestureRecognizerDelegate protocol so you're safe to implement your own gesture recognizers there. We're looking at making additional changes throughout AppKit to ensure that you are able to safely subclass and implement gesture recognisers without future grief for either of us.

That's great to know, thanks. I had missed that NSTextView and NSTableView's private implementation of the gesture recogniser delegate methods have underscores before them so don't get in the way.

Anyway, thanks again, your answers have been really informative and helpful and I now know how to approach my updates.

How to override NSTextView dragging behaviour without overriding mouseDown:?
 
 
Q