What is the distributional impact of low-skilled immigration to the US?

Dani Rodrik argues that maximizing the gains from trade liberalization should involve considerations of (1) efficiency, (2) second-best imperfections/externalities, and (3) distributional impact.

Rodrik argues that we should not prioritize liberalization of unskilled immigration to the US because it may cause “adverse movements in income distribution.” Unfortunately, Rodrik does not define what constitutes an adverse distributional effect.

I assume that Rodrik would not oppose a Pareto improvement that raised the real income of every member of the population but yielded disproportionate gains to the rich and therefore worsened the Gini coefficient (or other measures of distributional skew). Presumably, the concern about the distributional impact translates into something like a social welfare function that gives greater weight to the less well-off.

But allowing more low-skilled migrants to enter the United States would produce massive gains for those laborers. And immigrants from developing countries are frequently significantly poorer than low-skilled American workers. Significantly improving the welfare of those who are less well-off due to being born on the wrong side of an arbitrary line strikes me as a desirable distributional shift from a global perspective.

Interestingly, Rodrik has previously said that he is closer to Lant Pritchett than George Borjas on immigration:

George thinks the purely national perspective is the right one, and he figures the aggregate gains for the U.S. are small relative to the distributional costs, which makes this bad policy.  For my part, I believe cosmopolitan considerations should enter our calculus when the gains abroad (or to foreign nationals) are sufficiently large, which they would be with temporary labor flows.

So why does Rodrik now want to make low-skilled immigration liberalization a lower priority because it would make domestic unskilled workers worse off?