Place the potatoes in a large pot and cover with cold water; season the water generously with salt.
Bring to a boil, then reduce heat and simmer until the potatoes are tender when pierced with a fork, about 12–20 minutes.
While the potatoes cook, combine the chopped celery, mayonnaise, chopped chives, grated lemon zest, lemon juice, sugar, chopped tarragon, salt, and pepper in a large bowl; stir until smooth.
Drain the potatoes in a colander and let them cool completely, then halve or quarter to bite-sized pieces.
Add the cooled potato pieces to the dressing and toss gently until evenly coated; adjust seasoning if needed and serve.