dictionary
распаковка
m1 = dict(zip(df1.Id, df1.val))
m2 = dict(zip(df2.Id, df2.val))
pd.DataFrame([*{**m1, **m2}.items()], columns=['Id', 'val'])
Id val
0 1 5
1 3 7
2 4 5
3 7 2
4 9 2
Альтернативная форма
cols = ['Id', 'val']
m1 = dict(zip(*map(df1.get, cols)))
m2 = dict(zip(*map(df2.get, cols)))
pd.DataFrame([*{**m1, **m2}.items()], columns=cols)
get
m1 = dict(zip(df1.Id, df1.val))
m2 = dict(zip(df2.Id, df2.val))
f = lambda x: m2.get(x, m1.get(x, x))
pd.DataFrame([[x, f(x)] for x in {*df1.Id, *df2.Id}], columns=['Id', 'val'])
Id val
0 1 5
1 3 7
2 4 5
3 7 2
4 9 2