This whole article from Tweed in the City is fantastic.
“Pattern matching shouldn’t be about ad hoc tweaking. The typical approach is to throw things together out of brute impulse, then adjust by trial and error to salvage the resulting disaster. In my experience, you’re far more likely to succeed if you employ an organized decision-making process, a sequence of considered choices. To do that, you need solid principles.
When it comes to the four components that comprise the suit-based ensemble (suit, shirt, tie, and pocket square), there are only three principles you need to know.
1. Two or three are key.
Wearing only one pattern tends to look conspicuously anomalous, even though it’s widely considered a safe play. Adding a fourth pattern makes the balancing act exponentially more difficult. So, two or three are the sweet spot. The others can be done, but they carry a higher penalty for failure. Consider them advanced-level.
2. The pocket square stands alone.
The pocket square is a paradox: it’s a purely nonfunctional ornament, yet it must avoid immediately appearing that way. Books could be written about grappling with that problem. But for this discussion, it boils down to this basic fact: unlike colors and textures, patterns are intrinsically contrived, so the mind’s eye is especially quick to recognize and relate them. Hence, when your pocket square shares the same pattern with your shirt or tie, they are likely to appear paired. Then, the jig is up. The fantasy of nonchalance has been quashed before it even had a chance to takeoff. So, if your square has a pattern, make it distinct from everything else. A helpful tip: avoiding pocket squares with tie-like patterns makes life much easier.
3. “It’s the economydensity, stupid.”
Pattern variation is of the most important, yet misunderstood, concepts in the universe of classic menswear. It is one of the chief reasons outfits fail. Yet, it absolutely must be mastered.
Think of any given pattern as having two main attributes: type and density (color matters too, but that’s part of a whole other discussion). By type, I mean stripes versus check, versus herringbone, etc. Simple, right? Density is a little more complicated. It refers to the closeness and fineness of a pattern’s details. The more close and fine, the more dense. Scale partially influences density, but it’s not perfectly definitive. You can have a large-scale or non-repeating pattern that is very dense. Take a pocket square with a baroque scene for example, such as those from Rubinacci or Drake’s. They may only depict a single large subject, but if lavishly rendered, it may be tightly packed with different colors and shapes.
Ideally, all patterns in your outfit should vary appreciably from each in density. In fact, when wearing only two, consider it a hard rule. Why? Well, even if patterns are technically different in type, they tend to follow a few basic geometric patterns. For example, the little floral medallions popular on ties are arranged just like dots would be. So, if you were to wear a dotted pocket square of equal density, the patterns would read as conspicuously similar even though we call them different things. Another example is check and houndstooth. Nobody would ever confuse the two for each other, but realize that they both share the same grid-based, stacked square arrangement. Hence, two of similar density do not pair well.
Make no mistake, when it comes to pattern matching, density variation is far more controlling than pattern type. With sufficient variation in density, two items can share the same type of pattern quite easily–say, a candy-striped shirt and a wide-banded regimental tie. In contrast, two patterns of the same type and density should never, ever be paired. Matchingthree patterns of the same type is always likely to fail, but if anything is to save the day, it will be dynamic density variation. If two patterns are similarly dense, compensate by differentiating their pattern types as much as possible (different scales plus different geometric arrangements). Never wear three patterns of equivalent density.”