I tried looking to see if OpenCV or Intel Open Denoise did antialiasing, but no luck. What algorithm would you use if you had to make it from scratch?

Here is the code that I'm trying:

	// Load from file
	Mat aliased_mat = imread("aliased.png", IMREAD_UNCHANGED);

	if (aliased_mat.empty() || aliased_mat.channels() != 4)
	{
		cout << "aliased.png must be a 32-bit PNG" << endl;
		return -1;
	}

	Mat anti_aliased_mat(aliased_mat.rows, aliased_mat.cols, CV_8UC4);

	for (signed int i = 0; i < aliased_mat.cols; i++)
	{
		for (signed int j = 0; j < aliased_mat.rows; j++)
		{
			int horizontal_begin = i - 1;
			int vertical_begin = j - 1;

			if (horizontal_begin < 0)
				horizontal_begin = 0;

			if (vertical_begin < 0)
				vertical_begin = 0;

			int horizontal_end = i + 1;
			int vertical_end = j + 1;

			if (horizontal_end >= aliased_mat.cols)
				horizontal_end = aliased_mat.cols - 1;

			if (vertical_end >= aliased_mat.rows)
				vertical_end = aliased_mat.rows - 1;

			Vec4b pixelValue_centre = aliased_mat.at<Vec4b>(j, i);

			Vec4b pixelValue_left = aliased_mat.at<Vec4b>(j, horizontal_begin);
			Vec4b pixelValue_right = aliased_mat.at<Vec4b>(j, horizontal_end);

			Vec4b pixelValue_up = aliased_mat.at<Vec4b>(vertical_begin, i);
			Vec4b pixelValue_down = aliased_mat.at<Vec4b>(vertical_end, i);

			Vec4f horizontal_avg = (pixelValue_left + pixelValue_right);
			Vec4f vertical_avg = (pixelValue_up + pixelValue_down);

			Vec4f avg;
			avg[0] = (1.0 / 3.0) * (pixelValue_centre[0] + horizontal_avg[0] + vertical_avg[0]);
			avg[1] = (1.0 / 3.0) * (pixelValue_centre[1] + horizontal_avg[1] + vertical_avg[1]);
			avg[2] = (1.0 / 3.0) * (pixelValue_centre[2] + horizontal_avg[2] + vertical_avg[2]);
			avg[3] = 255;

			Vec4b avg_b;
			avg_b[0] = avg[0];
			avg_b[1] = avg[1];
			avg_b[2] = avg[2];
			avg_b[3] = avg[3];

			anti_aliased_mat.at<Vec4b>(j, i) = avg_b;
		}
	}

	imwrite("anti_aliased.png", anti_aliased_mat);

	return 0;

Regarding the code, calculating gradients from only 4 neighbors is usually low quality due to grid snapping. Using 8 neighbors prevents this already pretty well, but a larger filter kernel might work even better.

Pay attention to proper weighting. E.g. for a box filter to average 2x2 pixels, center pixel would have a weight of 4/16, edge neighbors a weight of 2/16, corner neighbors a weight of 1/16:

The same weights can be used to get a higher quality gradient, where the corner neighbours contribute only half as much as the edge neighbors.

Besides, likely you do not want to use a gradient, but rather direction of primary curvature.
This is rarely discussed, but a gradient becomes zero at the edge where you need it the most, while curvature direction has the largest magnitude at the edge as desired.

If you find an edge, you could average multiple samples along primary curvature direction (or perpendicular to the gradient), to smooth out staircase effects from polygon rasterization. I guess that's your idea with the code.
But it would not work for other sources of aliasing, e.g. noisy specular reflections.

Maybe you could try AMD Fidelity FX 1.0 (or 'FSR', not sure what's the name).
It's open source, and unlike other upscalers it requires no temporal input to work from multiple jittered frames. It's only spatial and thus works with still images. So you could use it to upscale the image, then sample it down to get AA.
And i guess there are spatial AI upscalers too, outside the gaming world.

Edit: There also was ‘FXAA’, if you remember :)

Thank you, sir.

I will add weighting.

Looking at your code more closely, i was leaping ahead regarding edge smoothing ideas.
Seems you just do a uniform blur. This could benefit from more neighbors and better weighting as well, but implementing it as a bilateral filter (two passes for horizontal / vertical directions) would make all this easier and faster. E.g. if you do (left+center*2+right)/4, then the same vertically, the result would be the exact same as with my given box filter weights.

However, i doubt a simple blur is what you want. Depends on the image and its problem ofc.

The blurring is for a full-colour shadow map of sorts. As is, it works pretty well. I’m going to leave it as is, using 4 neighbours.

Thank you again so much for all of your input! You’re the best, man.