C++ autodiff modifying const objects

[AyhamSaffar]

Following on from #2994 and possibly related to #2914:

#include <vector>
#include <cassert>

extern int enzyme_out;
extern int enzyme_const;

template < typename return_type, typename ... T >
return_type __enzyme_autodiff(void*, T ... );

struct DataStructure
{
    const std::vector<double> m_vec{};
    DataStructure(const std::vector<double>& V): m_vec{V} {};
    double multiplyAndSumBy(double num) const;
};

double DataStructure::multiplyAndSumBy(double num) const
{
    double sum{0};
    for (double element: m_vec)
        sum += num*element;
    return sum;
}

double calculate(double num, const DataStructure& DS) {return DS.multiplyAndSumBy(num);}

int main()
{
    const DataStructure DS{{1, 2, 3}};
    const DataStructure DSCopy{DS};
    double num{2};

    double grad{
        __enzyme_autodiff<double>((void*)calculate, enzyme_out, num, enzyme_const, &DS)
    };

    assert(DS.m_vec==DSCopy.m_vec && "const object mutated!");

    return 0;
}

Looks like enzyme is definitely modifying const objects passed to differentiated functions.

If there is anything i can do to help, I really like this library and would love to start contributing.

[vchuravy]

Related issue in Enzyme.jl EnzymeAD/Enzyme.jl#2526 enzyme_const is a strong assertion and not just a "treat this argument as inactive and complain if it is not".

I recall @wsmoses explanation being that for enzyme_const the primal pointer and the shadow pointer get collapsed, but I might misremember something.

Definitely needs some improved documentation, and I still think we should have something like enzyme_inactive instead of enzyme_const

[AyhamSaffar]

That sounds like a good idea. If you don't expect __enzyme_autodiff to modify const parameters, it can be really tricky to debug what is going wrong.

But also turns out that just having a dummy shadow pointer fixes the issue.

int main()
{
    const DataStructure DS{{1, 2, 3}};
    const DataStructure DSCopy{DS};
    double num{2};
    
    DataStructure DSGrad{{4, 5, 6}};
    double grad{
        __enzyme_autodiff<double>((void*)calculate, enzyme_out, num, enzyme_dup, &DS, DSGrad)
    };

    assert(DS.m_vec==DSCopy.m_vec && "const object mutated!");

    return 0;
}

Though i guess this does mean taking a performance hit.

[vchuravy]

You probably want your DSGrad to be all zero since it otherwise could show up in the gradient calculation.

[AyhamSaffar]

oh thanks. That would have also been really tricky to debug.