Fix shadow gaps at rounded CSD corners (#32)

[jEsuSdA]

Summary

Fixes #32 — windows with rounded Client-Side Decoration (CSD) corners show small gaps without shadow.

Problem

ARGB non-override_redirect windows (e.g. Thunar with GTK decorations) are classified as SHADOW_NOCENTER. The shadow center is made transparent by make_transparent_shadowcenter() using a rigid rectangular cutout of the exact window size. The rounded corners of the CSD frame are transparent (alpha=0), so the shadow underneath is also transparent, revealing whatever is behind the window.

Solution

Shrink the transparent center area by the window's _NET_FRAME_EXTENTS, plus a minimum CORNER_MARGIN of 4 px to cover typical rounded-corner radii (3–5 px) even when the window has no WM frame.

For CSD windows (Thunar, etc.)

• Frame extents are large (20–40 px).
• Transparent area is reduced, leaving the shadow visible under the decorative frame.
• The frame is opaque, so the shadow is hidden — except at the rounded corners, where the frame is transparent and the shadow now shows through. Bug fixed.

For transparent terminals / frameless ARGB windows

• Frame extents are 0, so the CORNER_MARGIN of 4 px applies.
• A small shadow halo remains at the edges. This is visually acceptable and prevents the gap at rounded corners.

Scope

• Only fastcompmgr.c is touched (4 function signatures updated).
• The win struct already carries left_width, right_width, top_width, bottom_width from get_frame_extents().
• Zero impact on the hot path (paint_all()). The change only runs once per window when the shadow is created.

Testing

• make clean && make produces zero warnings, zero errors.

[tycho-kirchner]

Thank you. I just merged this into the dev branch: 474dcdb

It will land in master soon.

[tycho-kirchner]

/* Shrink the transparent area by the frame extents so the shadow stays
   * visible under the decorative frame.  Always keep at least CORNER_MARGIN
   * pixels of shadow at each edge to cover typical rounded-corner radii
   * (3-5 px) even when the window has no WM frame (_NET_FRAME_EXTENTS == 0). */
  const int CORNER_MARGIN = 4;
  int margin_left   = left   > CORNER_MARGIN ? left   : CORNER_MARGIN;
  int margin_right  = right  > CORNER_MARGIN ? right  : CORNER_MARGIN;
  int margin_top    = top    > CORNER_MARGIN ? top    : CORNER_MARGIN;
  int margin_bottom = bottom > CORNER_MARGIN ? bottom : CORNER_MARGIN;

In the xfce4 notification, during fade out, this renders an ugly square shadow. Thus I am probably going to ditch the artificial margin. Do you have an example, where it is useful?

[jEsuSdA]

Yes, it is!

Sadly, the solution has some not-very-beautiful effects on some cases.

But it was the easy and simple way I found to solve without making huge changes in code.

In my earlier issue there are more info and you can see some screenshots: #32

Cheers!!!

[tycho-kirchner]

Let me be more specific.
For Thunar it is obvious that the white in the corner is wrong and should be fixed. As you say, thunar specifies some nonzero frame extents.

Your comment says

Always keep at least CORNER_MARGIN
pixels of shadow at each edge to cover typical rounded-corner radii
(3-5 px) even when the window has no WM frame (_NET_FRAME_EXTENTS == 0)

So, for instance, a transparent mate-terminal with _NET_FRAME_EXTENTS==0 looks like this when your artificial CORNER_MARGIN is active:

To me the gray stripe does not look visually appealing.

On the other hand, it looks like this after I removed the artificial CORNER_MARGIN (only for _NET_FRAME_EXTENTS==0):

I would argue, that the latter looks better. Thus, I would like to ask for a particular _NET_FRAME_EXTENTS==0 application that looks better with artificial CORNER_MARGIN.

[tycho-kirchner]

FYI: I found some more quirks.
I think I will drop the center-shadow soon.
Zoom's screen share frame can be detected by the Xshape extension. Adding code that detects such windows should be easy.
argb windows will re-gain there center shadow. The dropdown shadow behind transparent terminals is probably acceptable -- after all the shadow can be configured more opaque. For fullscreen windows the shadow could be omitted altogether,.

[jEsuSdA]

Now I understand better.

I think you're right, and I also think it's great news that you've found a better way to solve this.

Thank you so much for your interest, dedication... and patience with me. ;)