GH-149142: Fix decimal mpd_context_t.status race in free-threading

[LindaSummer]

Issue

GH-149142

Proposed Changes

Add mutex in free-threading build to guard the PyDecContextObject.ctx.status.
It solves the race in concurrent dec_addstatus and clear_flags.

Comment

For other flags like traps are not guarded in this PR.
I plan to update in later PRs with new test cases.

[LindaSummer]

During this patch, I find another race in decimal.DefaultContext.

In context_new, we may create a new context from decimal.DefaultContext.
But the decimal.DefaultContext could be updated in another thread.

cpython/Modules/_decimal/_decimal.c

Lines 1467 to 1469 in 1c7011d

	if (state->default_context_template) {
	*ctx = *CTX(state->default_context_template);
	}

So I think that we should have a way to get a snapshot of default_context_template just like the contextvar's HMAT in #146482.

Refer to our documentation we could see below:

This context is most useful in multi-threaded environments. Changing one of the fields before threads are started has the effect of setting system-wide defaults. Changing the fields after threads have started is not recommended as it would require thread synchronization to prevent race conditions.

I'm not sure if this case is valid refer to our documentation.
But even it is not suggested, I think that the TSAN may still need to be fixed.

If we plan to fix this one, I am appreciating to create a new issue and try to solve it.

Here is the demo reproduce code and TSAN report.

import decimal
import threading


N_THREADS = 8
ITERATIONS = 100_000


def run_concurrently(workers):
    barrier = threading.Barrier(len(workers))
    threads = [threading.Thread(target=worker, args=(barrier,))
               for worker in workers]
    for thread in threads:
        thread.start()
    for thread in threads:
        thread.join()


default_context = decimal.DefaultContext

def creator(barrier):
    barrier.wait()
    for _ in range(ITERATIONS):
        decimal.Context()

def mutator(barrier):
    barrier.wait()
    for i in range(ITERATIONS):
        default_context.prec = 9 + (i % 10)
        default_context.Emax = 999999 - (i % 10)
        default_context.Emin = -999999 + (i % 10)
        default_context.create_decimal("1.23456")
        default_context.clear_flags()

run_concurrently([creator] * N_THREADS + [mutator] * N_THREADS)

TSAN report.

SUMMARY: ThreadSanitizer: data race Modules/_decimal/_decimal.c:616 in dec_addstatus
==================
==================
WARNING: ThreadSanitizer: data race (pid=1872326)
  Write of size 4 at 0x7f21063640bc by thread T16:
    #0 dec_addstatus Modules/_decimal/_decimal.c:616 (_decimal.cpython-316t-x86_64-linux-gnu.so+0xf475)
    #1 PyDecType_FromCString Modules/_decimal/_decimal.c:2327 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
    #2 PyDecType_FromSequence Modules/_decimal/_decimal.c:2934 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
    #3 PyDec_FromObject Modules/_decimal/_decimal.c:3234 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
    #4 _decimal_Context_create_decimal_impl Modules/_decimal/_decimal.c:3292 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
    #5 _decimal_Context_create_decimal Modules/_decimal/clinic/_decimal.c.h:853 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
    #6 _Py_BuiltinCallFast_StackRef Python/ceval.c:815 (python3.16+0x3bfa1b)
    #7 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2420 (python3.16+0x3df9d5)
    #8 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
    #9 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
    #10 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
    #11 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
    #12 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
    #13 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
    #14 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x42729b)
    #15 context_run Python/context.c:728 (python3.16+0x42729b)
    #16 method_vectorcall_FASTCALL_KEYWORDS Objects/descrobject.c:421 (python3.16+0x1743fe)
    #17 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x151cfc)
    #18 PyObject_Vectorcall Objects/call.c:327 (python3.16+0x151cfc)
    #19 _Py_VectorCallInstrumentation_StackRefSteal Python/ceval.c:766 (python3.16+0x3bf555)
    #20 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1846 (python3.16+0x3cf120)
    #21 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
    #22 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
    #23 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
    #24 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
    #25 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
    #26 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
    #27 _PyVectorcall_Call Objects/call.c:273 (python3.16+0x154be8)
    #28 _PyObject_Call Objects/call.c:348 (python3.16+0x154fe1)
    #29 PyObject_Call Objects/call.c:373 (python3.16+0x154fe1)
    #30 thread_run Modules/_threadmodule.c:388 (python3.16+0x608d30)
    #31 pythread_wrapper Python/thread_pthread.h:234 (python3.16+0x5072d2)
    #32 <null> <null> (libtsan.so.2+0x4cf1a)

  Previous read of size 8 at 0x7f21063640b8 by thread T1:
    #0 context_new Modules/_decimal/_decimal.c:1468 (_decimal.cpython-316t-x86_64-linux-gnu.so+0xd6af)
    #1 type_call Objects/typeobject.c:2467 (python3.16+0x28839b)
    #2 _PyObject_MakeTpCall Objects/call.c:242 (python3.16+0x150dae)
    #3 _PyObject_VectorcallTstate Include/internal/pycore_call.h:142 (python3.16+0x151d61)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:129 (python3.16+0x151d61)
    #5 PyObject_Vectorcall Objects/call.c:327 (python3.16+0x151d61)
    #6 _Py_VectorCall_StackRefSteal Python/ceval.c:724 (python3.16+0x3bf125)
    #7 _PyEval_EvalFrameDefault Python/generated_cases.c.h:4362 (python3.16+0x3db2fc)
    #8 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
    #9 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
    #10 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
    #11 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
    #12 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
    #13 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
    #14 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x42729b)
    #15 context_run Python/context.c:728 (python3.16+0x42729b)
    #16 method_vectorcall_FASTCALL_KEYWORDS Objects/descrobject.c:421 (python3.16+0x1743fe)
    #17 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x151cfc)
    #18 PyObject_Vectorcall Objects/call.c:327 (python3.16+0x151cfc)
    #19 _Py_VectorCallInstrumentation_StackRefSteal Python/ceval.c:766 (python3.16+0x3bf555)
    #20 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1846 (python3.16+0x3cf120)
    #21 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
    #22 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
    #23 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
    #24 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
    #25 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
    #26 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
    #27 _PyVectorcall_Call Objects/call.c:273 (python3.16+0x154be8)
    #28 _PyObject_Call Objects/call.c:348 (python3.16+0x154fe1)
    #29 PyObject_Call Objects/call.c:373 (python3.16+0x154fe1)
    #30 thread_run Modules/_threadmodule.c:388 (python3.16+0x608d30)
    #31 pythread_wrapper Python/thread_pthread.h:234 (python3.16+0x5072d2)
    #32 <null> <null> (libtsan.so.2+0x4cf1a)

[LindaSummer]

Hi @devdanzin ,

Could you help take a review?
Please correct me if I made any mistake or misunderstanding.
Wish you a good day! ❤