Float Collinearity Causes Self-Intersecting Polygons
Naive line intersection using raw float equality failed on nearly collinear vertices, causing a self-intersecting polygon destroying three prototypes..
20+ years shipping performance-critical code where algorithms decide the bill. Drawn from code that ran under real load.
- ✓Solid grasp of fundamentals
- ✓Comfortable reading code examples
- ✓Basic production concepts
- Orientation test: cross product of vectors (q-p) x (r-p). Positive = CCW, Negative = CW, Zero = collinear. No division — exact for integer coordinates.
- Segment intersection: segments AB and CD intersect iff {A,B} straddle CD AND {C,D} straddle AB. Collinear cases require an on-segment bounding-box check.
- Intersection point: solve parametric line equations. Denominator near zero means parallel — check with epsilon.
- Performance: O(1) per test. No allocations, no division in orientation. Batch tests dominate sweep line algorithms.
- Production insight: floating-point precision is the #1 source of geometry bugs. Use integer coordinates or epsilon comparisons — never raw float equality.
- Biggest mistake: forgetting collinear endpoint cases. Naive implementations return wrong results when an endpoint lies exactly on the other segment.
This article tackles a specific, maddening failure mode in computational geometry: when floating-point precision causes line intersection algorithms to produce self-intersecting polygons. You're building a GIS system, a physics engine, or a PCB layout tool, and your polygon clipping or boolean operations suddenly generate degenerate shapes with edges that cross where they shouldn't.
The root cause is almost always float collinearity — three or more points that are mathematically collinear but, due to IEEE 754 rounding, register as nearly-but-not-quite collinear. This breaks the orientation test, the fundamental primitive that every line intersection algorithm relies on to determine whether a point lies left, right, or on a line.
When that test flips unpredictably, your sweep line algorithm or naive pairwise check produces phantom intersections or misses real ones, yielding polygons that intersect themselves.
The article walks through how line intersection algorithms actually work under the hood, starting with the orientation test using cross products and the determinant (the Shoelace formula's cousin). You'll see why segment intersection boils down to two orientation checks and a bounding box test, and how computing the actual intersection point via parametric lines or Cramer's rule introduces more floating-point error.
The naive O(n²) pairwise approach is explained as the baseline you should never use in production — it's a debugging tool, not a solution. The real workhorse is the sweep line algorithm (Bentley-Ottmann and its variants), which reduces complexity to O((n+k) log n) by sorting events and maintaining an active set.
But even sweep line fails when float collinearity corrupts the ordering of event points or the comparison of segment endpoints.
Where does this fit? If you're using robust geometry libraries like CGAL, JTS, or Shapely, they handle this with exact arithmetic (rational numbers or arbitrary precision) — but at a performance cost. For real-time or embedded systems where you can't afford that, you need epsilon-tolerant predicates and snap-rounding.
This article is for the engineer who has to implement their own intersection logic (e.g., in a game engine, CAD plugin, or custom geospatial tool) and needs to understand why their polygons are tearing apart. It's not for you if you can use a battle-tested library like Clipper2 or Boost.Geometry — those already solved this.
But if you're debugging why your homegrown algorithm produces self-intersecting output, this is the deep dive into the floating-point pathology that causes it.
Two line segments on a plane: do they cross? This sounds simple but requires careful handling of collinear points, endpoints, and floating-point precision. The cross product orientation test answers this in O(1) — and it is the building block of convex hull, polygon clipping, and sweep line algorithms.
Line intersection is the foundational primitive of computational geometry. Collision detection, polygon clipping, sweep line algorithms, and convex hull construction all reduce to one question: do two segments intersect? The cross product orientation test answers this in O(1) with no division, making it numerically robust and fast.
Production geometry code fails not on the common case but on edge cases: collinear overlapping segments, endpoints lying exactly on segments, and floating-point precision at near-parallel intersections. A single missed collinear check can corrupt a convex hull, create self-intersecting polygons, or cause collision detection to miss contacts.
A common misconception is that line equations are equivalent to the cross product approach. They are not. Line equations require division, which introduces floating-point error. The cross product uses only multiplication and subtraction — exact for integer coordinates and significantly more robust for floats.
How Line Intersection Algorithms Actually Work
Line intersection algorithms determine whether two line segments cross in 2D space and compute the exact intersection point. The core mechanic is solving the parametric equations of each segment: P = A + t(B - A) and Q = C + u(D - C). The intersection exists when 0 ≤ t ≤ 1 and 0 ≤ u ≤ 1, meaning both points lie within their respective segments. This is fundamentally an orientation test using cross products, not slope-intercept math.
In practice, the algorithm relies on orientation predicates: given three points, do they form a clockwise, counterclockwise, or collinear turn? Two segments intersect only if their orientations are opposite (or one is collinear and the other straddles). The cross product sign is the key — but floating-point precision makes exact collinearity detection unreliable. A tolerance (epsilon) of 1e-9 or 1e-12 is standard, but too large an epsilon introduces false positives.
Use this algorithm whenever you need to detect polygon self-intersection, compute boolean operations on shapes, or validate geometry in GIS or CAD systems. It's O(1) per segment pair, but naive O(n²) checks on all pairs in a polygon with n vertices can be slow. Spatial indexing (e.g., sweep-line, R-tree) reduces this to O(n log n). The real-world cost of a missed or false intersection is a corrupted polygon that breaks downstream rendering or physics engines.
Orientation Test — The Core Primitive
The orientation of three points p, q, r is determined by the cross product of vectors (q-p) and (r-p): - Positive: counterclockwise (left turn) - Negative: clockwise (right turn) - Zero: collinear
This is the single operation that underlies all segment intersection tests. The cross product formula is:
val = (q.y - p.y) (r.x - q.x) - (q.x - p.x) (r.y - q.y)
No division is involved — only multiplication and subtraction. This makes the test exact for integer coordinates and significantly more robust than line-equation approaches that require dividing by a potentially-zero denominator.
def orientation(p, q, r): """ Returns: 1 — counterclockwise (left turn) -1 — clockwise (right turn) 0 — collinear """ val = (q[1]-p[1])*(r[0]-q[0]) - (q[0]-p[0])*(r[1]-q[1]) if val > 0: return 1 if val < 0: return -1 return 0 print(orientation((0,0),(1,0),(0,1))) # 1 (CCW) print(orientation((0,0),(1,0),(2,0))) # 0 (collinear) print(orientation((0,0),(0,1),(1,0))) # -1 (CW)
- Cross product = 2 * signed area of triangle (p, q, r)
- Positive area = points are counterclockwise
- Zero area = points are collinear (triangle degenerates to a line)
- No division means no rounding error for integer inputs
Segment Intersection
Two segments AB and CD intersect if their orientations 'straddle' each other. Specifically, A and B must be on opposite sides of line CD, AND C and D must be on opposite sides of line AB. This is checked by verifying that orientation(A,B,C) != orientation(A,B,D) and orientation(C,D,A) != orientation(C,D,B).
Edge case: when one or more orientations are zero (collinear), the segments may still intersect if an endpoint lies on the other segment. The on_segment test checks this by verifying the collinear point falls within the bounding box of the segment.
def on_segment(p, q, r): """Does point q lie on segment pr (given collinearity)?""" return (min(p[0],r[0]) <= q[0] <= max(p[0],r[0]) and min(p[1],r[1]) <= q[1] <= max(p[1],r[1])) def segments_intersect(p1, q1, p2, q2) -> bool: """Do segments p1q1 and p2q2 intersect?""" o1 = orientation(p1, q1, p2) o2 = orientation(p1, q1, q2) o3 = orientation(p2, q2, p1) o4 = orientation(p2, q2, q1) if o1 != o2 and o3 != o4: return True # Collinear cases if o1 == 0 and on_segment(p1, p2, q1): return True if o2 == 0 and on_segment(p1, q2, q1): return True if o3 == 0 and on_segment(p2, p1, q2): return True if o4 == 0 and on_segment(p2, q1, q2): return True return False print(segments_intersect((1,1),(10,1),(1,2),(10,2))) # False (parallel) print(segments_intersect((10,0),(0,10),(0,0),(10,10))) # True (X pattern)
Line Intersection Point
Given that two lines intersect, the intersection point is computed by solving the parametric equations of both lines simultaneously. For lines through p1-p2 and p3-p4, the denominator is the cross product of the direction vectors. If the denominator is zero, the lines are parallel. The parameter t gives the position along the first line, and the intersection point is p1 + t*(p2-p1).
This function operates on infinite lines, not segments. To find the intersection of segments, first verify they intersect using segments_intersect, then compute the point. For near-parallel lines, the denominator approaches zero and the intersection point becomes numerically unstable — handle this with an epsilon check.
def line_intersection(p1, p2, p3, p4): """Find intersection point of lines through p1-p2 and p3-p4.""" x1,y1,x2,y2 = *p1, *p2 x3,y3,x4,y4 = *p3, *p4 denom = (x1-x2)*(y3-y4) - (y1-y2)*(x3-x4) if abs(denom) < 1e-10: return None # parallel t = ((x1-x3)*(y3-y4) - (y1-y3)*(x3-x4)) / denom x = x1 + t*(x2-x1) y = y1 + t*(y2-y1) return (x, y) print(line_intersection((0,0),(1,1),(0,1),(1,0))) # (0.5, 0.5)
- line_intersection: where do the infinite lines through these points cross?
- segments_intersect: do the finite segments between these points overlap?
- To get segment intersection point: call segments_intersect first, then line_intersection
- Calling line_intersection alone gives false positives for non-overlapping segments
The Naive Approach — Pairwise Hell
You could check every segment against every other segment. That's O(n²). For small datasets—say, under 50 segments—it works fine. For anything larger, it's a death sentence.
The logic itself is simple: for each pair, run the orientation test on both endpoints. If the orientations are opposite (or collinear with overlapping ranges), they intersect. That check is O(1). So the whole thing is O(n²) time, O(1) space. No mystery.
Here's the truth: the naive approach isn't stupid, it's honest. It tells you exactly when you're in over your head. If your geometry engine handles 100K segments and you use this, you'll be watching a monitor go orange at 2 AM. Sweep line exists for a reason.
// io.thecodeforge — dsa tutorial public class PairwiseIntersection { static boolean onSegment(int[] p, int[] q, int[] r) { return q[0] <= Math.max(p[0], r[0]) && q[0] >= Math.min(p[0], r[0]) && q[1] <= Math.max(p[1], r[1]) && q[1] >= Math.min(p[1], r[1]); } static int orientation(int[] p, int[] q, int[] r) { int val = (q[1] - p[1]) * (r[0] - q[0]) - (q[0] - p[0]) * (r[1] - q[1]); if (val == 0) return 0; return (val > 0) ? 1 : 2; } static boolean intersect(int[] a1, int[] a2, int[] b1, int[] b2) { int o1 = orientation(a1, a2, b1); int o2 = orientation(a1, a2, b2); int o3 = orientation(b1, b2, a1); int o4 = orientation(b1, b2, a2); if (o1 != o2 && o3 != o4) return true; if (o1 == 0 && onSegment(a1, b1, a2)) return true; if (o2 == 0 && onSegment(a1, b2, a2)) return true; if (o3 == 0 && onSegment(b1, a1, b2)) return true; if (o4 == 0 && onSegment(b1, a2, b2)) return true; return false; } public static void main(String[] args) { int[][] seg1 = {{1, 5}, {4, 5}}; int[][] seg2 = {{2, 5}, {10, 1}}; System.out.println(intersect(seg1[0], seg1[1], seg2[0], seg2[1])); } }
Sweep Line — The Only Algorithm You Need
The sweep line algorithm changes the game from O(n²) to O(n log n). Here's how it works: imagine a vertical line sliding from left to right across your plane. As it moves, it maintains a list of 'active' segments—those currently crossing the sweep line, sorted by their Y-coordinate at the current X.
The key insight: intersections only happen between segments that are neighbors in this sorted order. So instead of checking every pair, you only check pairs that become adjacent as the sweep line passes. When a new segment enters, you test it against its immediate neighbors. When one leaves, you test the neighbors against each other.
You need a balanced BST or a similar structure for the active set. In Java, that's TreeMap with a custom comparator—but careful, floating point comparisons will bite you. Use integer arithmetic or epsilon-tolerant comparisons.
For line segments (not infinite lines), you also need to handle endpoint events: when the sweep line hits a left endpoint, insert the segment; when it hits a right endpoint, remove it. Insertion and neighbor queries are O(log n) each.
// io.thecodeforge — dsa tutorial import java.util.*; public class SweepLineEngine { static class Event implements Comparable<Event> { int x, y, segIdx, type; // 0 = left, 1 = right Event(int x, int y, int idx, int t) { this.x = x; this.y = y; this.segIdx = idx; this.type = t; } public int compareTo(Event o) { if (this.x != o.x) return this.x - o.x; return this.y - o.y; } } public static void main(String[] args) { // dummy: shows skeleton. Real impl needs comparator with sweep X context System.out.println("Sweep line skeleton ready"); } }
Orientation of Three Points — The 2D Gut Check
Orientation is the foundation of everything here. Given three points A, B, C, orientation tells you if C is to the left, right, or collinear with the directed line AB. It's a cross product sign test, no trigonometry needed.
The formula: (B.y - A.y) (C.x - B.x) - (B.x - A.x) (C.y - B.y). If positive -> counterclockwise (left turn). Negative -> clockwise (right turn). Zero -> collinear.
Why this works: the cross product of vectors AB and BC gives twice the signed area of triangle ABC. If the area is positive, C is left of AB. That's it. No floating point errors if you use integers.
Collinear cases are the ugly part. When orientation is zero, you need an extra containment check: does point C lie within the bounding box of AB? That's onSegment(). Without it, you'll miss intersections where segments share an endpoint or overlap partially.
// io.thecodeforge — dsa tutorial public class OrientationTest { static int orientation(int[] a, int[] b, int[] c) { int cross = (b[1] - a[1]) * (c[0] - b[0]) - (b[0] - a[0]) * (c[1] - b[1]); if (cross > 0) return 1; // counterclockwise if (cross < 0) return 2; // clockwise return 0; // collinear } static boolean onSegment(int[] a, int[] b, int[] c) { return b[0] <= Math.max(a[0], c[0]) && b[0] >= Math.min(a[0], c[0]) && b[1] <= Math.max(a[1], c[1]) && b[1] >= Math.min(a[1], c[1]); } public static void main(String[] args) { int[] a = {0, 0}, b = {10, 10}, c = {5, 5}; int orient = orientation(a, b, c); System.out.println("Orientation: " + orient + " (0=collinear)"); System.out.println("On segment: " + onSegment(a, b, c)); } }
Graham Scan — Convex Hull in O(n log n)
Why care? Line intersection algorithms often assume you already know which segments are relevant. The Graham Scan eliminates irrelevant points by computing the convex hull — the smallest convex polygon containing all points. This is foundational: any line that doesn't intersect the hull never intersects the points inside it. Graham Scan works by first finding the lowest-y point (pivot), then sorting all other points by polar angle relative to that pivot. It scans the sorted list, maintaining a stack. For each new point, it checks orientation of the last two stack points and the new point — if the turn is clockwise (right turn), the middle point is inside the hull and gets popped. Only counter-clockwise turns stay. The result is a tight boundary you can feed directly into sweep-line algorithms. The critical nuance: collinear points on hull edges require special handling — either keep them for precision or discard them depending on whether you need minimal hull. This single pass after sorting gives O(n log n) total, beating naive pairwise methods.
// io.thecodeforge — dsa tutorial import java.util.*; class Point { int x, y; Point(int x, int y) { this.x = x; this.y = y; } } public class GrahamScan { static int orientation(Point p, Point q, Point r) { int val = (q.y - p.y) * (r.x - q.x) - (q.x - p.x) * (r.y - q.y); if (val == 0) return 0; // collinear return (val > 0) ? 1 : 2; // cw or ccw } static List<Point> convexHull(Point[] pts) { int n = pts.length; if (n < 3) return new ArrayList<>(); Arrays.sort(pts, (a, b) -> a.y != b.y ? a.y - b.y : a.x - b.x); Arrays.sort(pts, 1, n, (a, b) -> { int o = orientation(pts[0], a, b); if (o == 0) return (dist(pts[0], a) - dist(pts[0], b)) > 0 ? 1 : -1; return (o == 2) ? -1 : 1; }); Stack<Point> stack = new Stack<>(); stack.push(pts[0]); stack.push(pts[1]); for (int i = 2; i < n; i++) { while (stack.size() > 1 && orientation(stack.get(stack.size()-2), stack.peek(), pts[i]) != 2) stack.pop(); stack.push(pts[i]); } return new ArrayList<>(stack); } }
Chand–Kapur Algorithm — Avoiding Degenerate Sorting Perils
Why this matters? The Graham Scan's polar sort fails when points share the same angle (collinear with pivot) or when the convex hull has multiple collinear boundary points. Chand–Kapur (also called the Tentative Hull algorithm) fixes this by skipping the angle sort entirely. Instead, it builds the hull incrementally by traversing the sorted-by-x points and maintaining an upper and lower chain separately. For each chain, you push points and check orientation — if a right turn occurs, you pop the middle point. This avoids the collinear-angle ambiguity because you process points in monotonic x-order. The critical insight: process left-to-right for the upper hull, right-to-left for the lower hull. No angle computation, no floating-point. This makes Chand–Kapur more robust for integer coordinates and avoids edge-case crashes common in competitive programming. Complexity remains O(n log n) from sorting by x, but the constant is lower because you skip the angle comparison. Use this when your input contains many collinear points or when you need exact integer output without rounding errors.
// io.thecodeforge — dsa tutorial import java.util.*; class Point { int x, y; Point(int x, int y) { this.x = x; this.y = y; } } public class ChandKapur { static int cross(Point o, Point a, Point b) { return (a.x - o.x) * (b.y - o.y) - (a.y - o.y) * (b.x - o.x); } static List<Point> hull(Point[] pts) { Arrays.sort(pts, (a, b) -> a.x != b.x ? a.x - b.x : a.y - b.y); List<Point> lower = new ArrayList<>(); for (Point p : pts) { while (lower.size() >= 2 && cross(lower.get(lower.size()-2), lower.get(lower.size()-1), p) <= 0) lower.remove(lower.size()-1); lower.add(p); } List<Point> upper = new ArrayList<>(); for (int i = pts.length-1; i >= 0; i--) { while (upper.size() >= 2 && cross(upper.get(upper.size()-2), upper.get(upper.size()-1), pts[i]) <= 0) upper.remove(upper.size()-1); upper.add(pts[i]); } lower.addAll(upper.subList(1, upper.size()-1)); return lower; } }
Self-Intersecting Polygon in CAD Export Crashes CNC Machine
- Floating-point equality in geometry is never safe. Always use epsilon comparisons or exact arithmetic.
- Polygon simplification algorithms can produce self-intersecting output if the intersection predicate is imprecise.
- Post-condition validation (checking that output polygons are simple) catches bugs that the algorithm itself misses.
- Test with adversarial inputs: near-collinear vertices, near-parallel edges, degenerate triangles. Real CAD data is full of these.
Print orientation values for all four point triples to verify they are near-zeroCheck if on_segment bounding-box test uses raw float or epsilon comparisonAdd debug print: print(f'o1={o1} o2={o2} o3={o3} o4={o4}') before returnTest with segments sharing exactly one endpoint: ((0,0),(1,1)) and ((1,1),(2,0))Print denom value: should be > EPS for valid intersectionIf denom < EPS, treat as parallel — return None instead of dividingProfile: count total segment pairs being testedImplement Bentley-Ottmann sweep line to reduce to O((n+k) log n)| Approach | Arithmetic | Handles Collinear | Handles Endpoint | Performance | Best For |
|---|---|---|---|---|---|
| Cross Product Orientation | Multiplication + subtraction only | Yes (with on_segment) | Yes (4 collinear checks) | O(1) per test, no division | General segment intersection, convex hull, sweep line |
| Line Equation (y = mx + b) | Division required | No (slope undefined for vertical) | Requires special cases | O(1) but division overhead | Simple horizontal/vertical cases only |
| Parametric (p + t*d) | Multiplication + division | Yes (denom = 0 check) | Yes (t in [0,1] check) | O(1) per test | Computing intersection POINTS, ray casting |
| Exact Rational Arithmetic | Integer operations on fractions | Exact — no epsilon | Exact — no epsilon | Slower (GCD per operation) | Safety-critical CAD, formal verification, financial geometry |
| Sweep Line (Bentley-Ottmann) | Cross product at core | Yes | Yes | O((n+k) log n) for n segments | Batch intersection detection on large segment sets |
| File | Command / Code | Purpose |
|---|---|---|
| orientation.py | def orientation(p, q, r): | Orientation Test |
| segment_intersection.py | def on_segment(p, q, r): | Segment Intersection |
| intersection_point.py | def line_intersection(p1, p2, p3, p4): | Line Intersection Point |
| PairwiseIntersection.java | public class PairwiseIntersection { | The Naive Approach |
| SweepLineEngine.java | public class SweepLineEngine { | Sweep Line |
| OrientationTest.java | public class OrientationTest { | Orientation of Three Points |
| GrahamScan.java | class Point { int x, y; Point(int x, int y) { this.x = x; this.y = y; } } | Graham Scan |
| ChandKapur.java | class Point { int x, y; Point(int x, int y) { this.x = x; this.y = y; } } | Chand–Kapur Algorithm |
Key takeaways
Practice These on LeetCode
Interview Questions on This Topic
Frequently Asked Questions
Two approaches: (1) Use integer coordinates and exact arithmetic — cross product is exact for integers. (2) Use epsilon comparisons — treat |val| < eps as collinear. For production computational geometry (CAD, GIS), exact arithmetic libraries like CGAL or Shapely handle this robustly.
Line intersection finds where two infinite lines cross — it returns a point even if the segments between the given points do not overlap. Segment intersection checks if two finite segments share any point. Always use segment intersection for collision detection and polygon operations.
Slope computation requires division, which introduces floating-point error and fails on vertical lines (infinite slope). The cross product uses only multiplication and subtraction — no division, no special cases for vertical lines, and exact results for integer coordinates.
Epsilon should be scaled to your coordinate range. For unit-scale coordinates, 1e-10 works. For pixel coordinates (0-4096), use 1e-6. For GPS coordinates, use 1e-12. A rule of thumb: EPS should be roughly (coordinate_range^2) * 1e-15 to account for the cross product being a squared-area quantity.
It is an O((n+k) log n) algorithm for finding all k intersections among n segments, where naive pairwise checking is O(n^2). It sweeps a vertical line from left to right, maintaining an ordered set of active segments and an event queue of endpoints and intersection points. The orientation test is used to order segments and detect new intersections.
20+ years shipping performance-critical code where algorithms decide the bill. Drawn from code that ran under real load.
That's Geometry. Mark it forged?
5 min read · try the examples if you haven't