Rendered at 04:26:37 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
gavinray 7 hours ago [-]
Funny story about drag-and-drop:
At my very first job, we had a weekly meeting to assign tickets and estimates.
Our non-technical PM wrote "drag-and-drop" on a whiteboard, and then my name.
"Gavin, how long do you think this will take?"
"All week. At least. Maybe longer?"
PM laughed, then realized I wasn't making a joke.
"What do you mean 'all week'? That's ridiculous, do I need to give this to someone else?"
"Well, you could, but I wouldn't count on you getting it any sooner."
Appalled, he exclaims: "It's just drag-and-drop!"
After 10-15 minutes of a room full of engineers trying to explain the difficulty, he gave up with an unhappy "Fine."
The phrase "it's just drag-and-drop" became an inside joke for the rest of my time there...
freecodeio 6 hours ago [-]
I think non-tech people think that drag and drop is some sort of emergent property because you're pressing the mouse and holding it pressed.
adastra22 6 hours ago [-]
Dumb question: why? It’s been a long time since I touched GUI code, but I thought it gave you a message with the URI to the file. How is this significantly different than file-open?
gavinray 5 hours ago [-]
This was circa 2010 or so, and it was a web app. So pre-ES6 JavaScript, before React.
Nowadays you would just "npm install drag-and-drop" but back then you often had to hand-roll solutions.
If anyone can dig up an ancient library on GitHub for drag-and-drop back in those days, I'd imagine it would be several thousand lines long, not to mention the absolute nightmare that testing something like this with a proper framework and accompanying testing library.
adastra22 4 hours ago [-]
Ah, I was thinking desktop applications.
anonymous908213 4 hours ago [-]
Hit-test where the mouse-down event is fired, hit-test where the mouse-up event is fired. What the hell do you need a library for?
> If anyone can dig up an ancient library on GitHub for drag-and-drop back in those days, I'd imagine it would be several thousand lines long,
I bet, the JS devs who do write code overcompensate for the ones who can only stitch together libraries. I suppose because the expectation is that the library consumer is incapable of writing their own code extending it, so the library itself must support every possible combination of use cases for a million different users, even though the code any individual consumer actually needs is more like 100 loc.
gavinray 3 hours ago [-]
> What the hell do you need a library for?
On the frontend, you have to re-calculate the display state for all the possible drop positions, in-between transitions, and drop-end-snap-back.
Think of a grid of cards, and all the possible drag, hover, and drop state transitions it can be in with any other card, and the animations required to make it look nice
Now also consider that you need to persist the final state on the backend, and handle things like rejection, cancellation, failed updates/errors.
It becomes a massive state chart and web of complexity
anonymous908213 3 hours ago [-]
I mean, maybe we're missing context here. Is that what your boss was actually asking for? 99% of web applications (or, hell, desktop applications) don't have ultra-interactive, fully-responsive-animation drag-and-drop. If he was asking for functional drag-and-drop and the team was implementing top 1% cosmetic drag-and-drop, that's certainly one avenue for a mismatch in expectations.
dofm 1 hours ago [-]
What does that look like mid drag, and how?
If you are dragging within the page, then the draggable “ghost” is your problem too (it’s not automatic). As is deciding whether the ghost should affirmatively snap out of and into the source and target containers (that is, whether the destination container should appear to “make room” for the ghost at some transitional point), whether there’s a transition on that, etc.
Drag and drop in desktop non-web GUIs have all sorts of visual affordances that users expect for consistency that are your job in HTML.
I was once quite up-to-date with raw frontend work and I tried to implement this; it’s actually not trivial.
anonymous908213 1 hours ago [-]
> draggable ghost
Okay, you've added a requirement. Poll the current cursor position after the first hit-test sets the dragging state and call a function to render your object with X% opacity at a higher Z level than anything else. Your 30 minute job has become a 60 minute job.
> As is deciding whether the ghost should affirmatively snap out of and into the source and target containers (that is, whether the destination container should appear to “make room” for the ghost at some point), whether there’s a transition on that,
Most applications don't go this far, including your operating system like Windows that sets baseline UX expectations, so you're well beyond the default expectation for drag-and-drop. If someone asks you to implement drag-and-drop without specifying they want all the bells and whistles, and you respond it's going to take you a week because you're planning to go all-out beyond necessary business requirements, they'd have a valid reason to be like "WTF mate?". Also this is still pretty paltry and not a week's worth of work.
dofm 1 hours ago [-]
Did you ever actually do this work?
Do you remember what browser testing was like?
Remember the quirks of browsers’ different understanding of transparency, of z-index, of events, of DOM compliance?
I think the suggestion that getting it right could take a week in the context of a particular application is actually not particularly hyperbolic.
Mind you, I was doing cross-browser testing long enough ago that I am one of those people who can say they independently invented AJAX in-house.
anonymous908213 54 minutes ago [-]
I've done it a dozen times in gamedev, where interfaces and interactivity are routinely built from scratch without any help of a framework. I didn't work on web applications in 2010, but even with the clusterfuck that the web is I find it difficult to believe it was that hard to do.
dofm 44 minutes ago [-]
It could be absolutely exasperating, even in 2010 (especially then since browser support for touch events was still shaking out).
The problem is that even without a framework, you didn’t have a blank slate (which would have been better). You had a moving target of blank slates, you had platform differences, you had drag vs touch-drag, entrenched older browsers and the simple frustration of testing on an array of real devices, all sorts.
Now, if you were talking about 2015, then it starts to get a bit flatter.
ehnto 3 hours ago [-]
I built a drag and drop content CMS/website builder in native Javascript years ago. The browser APIs are really cumbersome. The other issue is more UX than technical, there are lots of questions to answer. What are you dragging, what is it hovering over actually, what is the intention once it's dropped etc.
mcculley 6 hours ago [-]
There are more possible actions than opening files.
giwook 5 hours ago [-]
I love when non-technical people think they know better than the technical people about technical things...
timcobb 6 hours ago [-]
good thing you had backup. rough hahaha.
btown 8 hours ago [-]
Oh, this is very, very cool, and very, very timely!
IMO one of the remaining gaps of agentic coding is that it's hard for even multimodal LLMs to check interactive motion effects for correctness and "feel." Like, sure, they can take screenshots of a few snapshotted states, but in terms of how things react as underlying nodes move and (potentially) reflow. How do you QA the feel of moving your wrist rapidly in the context of nested zones?
The answer is that you avoid the need to: you have a (nearly-provably) correct foundation that ensures that behavior is predictable regardless of state, and just render accordingly.
The paper's approach is incredible here: simultaneous optimistic rendering of multiple possible states, and a composable algebra (a small combinator language) over how drop zones and distance are derived - allowing for an extremely high-level interface in practice. Everything feels snappy and right, and the code you write is about as minimal as you can get - though the paper notes that state management is somewhat complex. But this seems solvable!
Once this is ported to HTML, it'll become an incredible part of the modern dev toolkit. I'm very excited for what this means.
disconcision 7 hours ago [-]
i have a basic html version working with solid, using basically the FLIP technique to get the measurements. dunno in what form we might try to release it though, its a bit awkward to generalize over existing frameworks
edflsafoiewq 14 minutes ago [-]
The Twisted Trees example is really cool. I would not have been able to implement it that well. After playing with it, I think at the most basic the underlying principle is:
> Write your interface as a function of continuous variables, ui = ui(x1,x2,...) = ui(X). When dragging an element, compute the barycentric coordinates λ₁,λ₂,λ₃ of the cursor between the three nearest dropzones. Then render by interpolating the ui parameters between their values at those three dropzones, ui(λ₁X₁ + λ₂X₂ + λ₃X₃).
sourdecor 7 hours ago [-]
I think this is a really cool project, but I also really appreciate the introduction to Hazel[0].
Hazel is such a god tier system, all the respect. Taking programming and making representing things in this live system good is such a huge bridge to a better programming, programming that welcomes the world in.
Thanks for the mention. I ran across Declarative Drawing & I really want excuses to use this. But I don't remember Hazel from the video I watched in it!
codedokode 4 hours ago [-]
Interesting idea, but look too complicated for me. What if there are 1000 places where an element can be dropped? What if we select and drag several elements at a time?
Also this approach seems to introduce new complicated concepts like "composables" and even a DSL. Probably writing custom code for Drag-and-drop would be easier that figuring out how this works. I read the page and did not understand much. For example, this phrase is difficult to understand: "Rather than programming all that visual behavior directly, you simply say, for each element you want to make draggable, which future states of your application dragging it should lead to.". Not "simply" at all. Writing new code would be easier especially with LLM.
nostrebored 3 hours ago [-]
The idea is that you can describe the predicates, producers, and goals. If I’m understanding correctly. This is how a ton of stuff works: excel, your IaC, etc.
Here the goal can be a render state and it is the solvers problem to figure out what should be there.
So you define these things very atomically, but the behavior automagically finds a valid render state (usually through converting things into a standard planner or solver language, haven’t dived into this repo yet)
gobdovan 6 hours ago [-]
Appreciable, but what is the novelty exactly? The API design? I see the paper doesn't call out Sketchpad or any of Ivan Sutherland's work anywhere in the paper, neither ThingLab or Juno(-2). They do call out Apparatus, which already introduced speculative rendering. The work also doesn't seem to generate valid transforms, which you have to supply yourself. Discovering valid transforms or solving a more general inverse rendering problem would have been much cooler research.
dwoldrich 29 minutes ago [-]
This API feels as significant to me as Reactive Extensions did when Netflix first debuted it. Thank you for this research! I have a lot to think about!
amelius 7 hours ago [-]
Does this also work when the search space becomes large?
I fear this might lead to an exponential blowup.
(The examples shown are all tiny)
jnpnj 5 hours ago [-]
I didn't read the paper yet, but do you they do space partitioning based on events direction ?
kkotak 8 hours ago [-]
Nice work! Good to see that someone is still working on making the Human Computer interface that has been around for while and used extensively in many applications more human. Thank you.
disconcision 7 hours ago [-]
an author here (andrew). down to answer questions though it might be a bit until i get to them
sebmellen 7 hours ago [-]
All I have to say is thank you for doing this! This is truly super cool. It’s a rare kind of gem to see thoughtful work like this in our new AI era.
rao-v 6 hours ago [-]
Someone else asked something similiar but basically, how does / should this work when candidate drop targets are many e.g. when dragging a first or second rectangle onto a page in Figma when no relationships have obviously been set.
disconcision 2 hours ago [-]
not 100% sure this is what you're asking but for continuous degrees of freedom i.e. numeric model parameters we have the 'vary' operator, which is used in all our canvas-type examples. more in the paper (section 5 under the numerical optimization header). in the trivial case this allows for unconstrained positioning
terabytest 7 hours ago [-]
Out of curiosity, does your library support physical springs for animation between states? That’d be a great touch.
disconcision 2 hours ago [-]
so far the library is history-independent wrt transitions, which is a nice simplification but does preclude more physics-y interactions.
michalsustr 6 hours ago [-]
Very cool demo! One idea I had in the back of my mind is extending grammar of graphics for interactive visualisations. What you did might be a good way how to approach this! Did you think about doing something like this btw.?
nostrebored 3 hours ago [-]
We will continue to reinvent dataflow and planning problems until the end of time; you will be surprised how many fundamental pieces of software use this paradigm
gobdovan 1 hours ago [-]
technically all of them since hardware itself is basically a dataflow system
cootsnuck 6 hours ago [-]
Very neat, looking forward to playing around with this. We are sorely in need of of more exploration into UX/UI, HCI of the current era of digital tools.
O_H_E 7 hours ago [-]
Anyone have an idea of what stack/technique would be used to accomplish a task that this is widely superior for. The work seems cool but I'm having trouble imagining what is it replacing.
Edit: nvm, answered by the Background section of the paper
leecommamichael 7 hours ago [-]
Speculative rendering sounds awfully expensive.
pyrolistical 6 hours ago [-]
It’s just rendering? I don’t see why it would be more expensive
leecommamichael 2 hours ago [-]
Rendering in this context means producing an element tree, or a textual description of one, which is capable of firing off many little (slow) allocations. The implication being that this will not scale to a large number of nodes without slogging the machine unduly.
gitgud 8 hours ago [-]
Wow that is an elegant way to do it. This could create a renaissance of drag n drop interactions for the web
gandreani 8 hours ago [-]
This is revolutionary! I wonder if there's any similar prior art.
The Hazel programming env is dope. I can already imagine using it to explain refactorings to new programmers.
The twisted tree diagrams is also really neat. Is that a new idea as well? I got the beginnings of an idea where it might be a good diagram to display tangled code paths.
Darky 9 hours ago [-]
Fantastic work!
The interactive demos are incredibly satisfying to play with, visually delightful, and genuinely educational.
petters 9 hours ago [-]
Agreed. It’s really good!!
fitsumbelay 4 hours ago [-]
all links and buttons in document header are also draggable demonstrations of the concept. very cool
Drag-and-drop is a user interface feature. More often we need drag tools for DAG (Directed Acyclic Graph), which is an execution architecture. Most work involves dependencies: something must happen before something else can happen. I didn't see DAG here.
At my very first job, we had a weekly meeting to assign tickets and estimates.
Our non-technical PM wrote "drag-and-drop" on a whiteboard, and then my name.
"Gavin, how long do you think this will take?"
"All week. At least. Maybe longer?"
PM laughed, then realized I wasn't making a joke.
"What do you mean 'all week'? That's ridiculous, do I need to give this to someone else?"
"Well, you could, but I wouldn't count on you getting it any sooner."
Appalled, he exclaims: "It's just drag-and-drop!"
After 10-15 minutes of a room full of engineers trying to explain the difficulty, he gave up with an unhappy "Fine."
The phrase "it's just drag-and-drop" became an inside joke for the rest of my time there...
Nowadays you would just "npm install drag-and-drop" but back then you often had to hand-roll solutions.
If anyone can dig up an ancient library on GitHub for drag-and-drop back in those days, I'd imagine it would be several thousand lines long, not to mention the absolute nightmare that testing something like this with a proper framework and accompanying testing library.
> If anyone can dig up an ancient library on GitHub for drag-and-drop back in those days, I'd imagine it would be several thousand lines long,
I bet, the JS devs who do write code overcompensate for the ones who can only stitch together libraries. I suppose because the expectation is that the library consumer is incapable of writing their own code extending it, so the library itself must support every possible combination of use cases for a million different users, even though the code any individual consumer actually needs is more like 100 loc.
Think of a grid of cards, and all the possible drag, hover, and drop state transitions it can be in with any other card, and the animations required to make it look nice
Now also consider that you need to persist the final state on the backend, and handle things like rejection, cancellation, failed updates/errors.
It becomes a massive state chart and web of complexity
If you are dragging within the page, then the draggable “ghost” is your problem too (it’s not automatic). As is deciding whether the ghost should affirmatively snap out of and into the source and target containers (that is, whether the destination container should appear to “make room” for the ghost at some transitional point), whether there’s a transition on that, etc.
Drag and drop in desktop non-web GUIs have all sorts of visual affordances that users expect for consistency that are your job in HTML.
I was once quite up-to-date with raw frontend work and I tried to implement this; it’s actually not trivial.
Okay, you've added a requirement. Poll the current cursor position after the first hit-test sets the dragging state and call a function to render your object with X% opacity at a higher Z level than anything else. Your 30 minute job has become a 60 minute job.
> As is deciding whether the ghost should affirmatively snap out of and into the source and target containers (that is, whether the destination container should appear to “make room” for the ghost at some point), whether there’s a transition on that,
Most applications don't go this far, including your operating system like Windows that sets baseline UX expectations, so you're well beyond the default expectation for drag-and-drop. If someone asks you to implement drag-and-drop without specifying they want all the bells and whistles, and you respond it's going to take you a week because you're planning to go all-out beyond necessary business requirements, they'd have a valid reason to be like "WTF mate?". Also this is still pretty paltry and not a week's worth of work.
Do you remember what browser testing was like?
Remember the quirks of browsers’ different understanding of transparency, of z-index, of events, of DOM compliance?
I think the suggestion that getting it right could take a week in the context of a particular application is actually not particularly hyperbolic.
Mind you, I was doing cross-browser testing long enough ago that I am one of those people who can say they independently invented AJAX in-house.
The problem is that even without a framework, you didn’t have a blank slate (which would have been better). You had a moving target of blank slates, you had platform differences, you had drag vs touch-drag, entrenched older browsers and the simple frustration of testing on an array of real devices, all sorts.
Now, if you were talking about 2015, then it starts to get a bit flatter.
Paper here: https://declarative-dragging.github.io/D2.pdf
IMO one of the remaining gaps of agentic coding is that it's hard for even multimodal LLMs to check interactive motion effects for correctness and "feel." Like, sure, they can take screenshots of a few snapshotted states, but in terms of how things react as underlying nodes move and (potentially) reflow. How do you QA the feel of moving your wrist rapidly in the context of nested zones?
The answer is that you avoid the need to: you have a (nearly-provably) correct foundation that ensures that behavior is predictable regardless of state, and just render accordingly.
The paper's approach is incredible here: simultaneous optimistic rendering of multiple possible states, and a composable algebra (a small combinator language) over how drop zones and distance are derived - allowing for an extremely high-level interface in practice. Everything feels snappy and right, and the code you write is about as minimal as you can get - though the paper notes that state management is somewhat complex. But this seems solvable!
Once this is ported to HTML, it'll become an incredible part of the modern dev toolkit. I'm very excited for what this means.
> Write your interface as a function of continuous variables, ui = ui(x1,x2,...) = ui(X). When dragging an element, compute the barycentric coordinates λ₁,λ₂,λ₃ of the cursor between the three nearest dropzones. Then render by interpolating the ui parameters between their values at those three dropzones, ui(λ₁X₁ + λ₂X₂ + λ₃X₃).
[0]: https://hazel.org/
In huge "Stop Drawing Dead Fish" / "Stop Writing Dead Programs" ways. Which I'm such a fan of. Asked Astra to chase down references on this a couple weeks ago, quite enjoyed the result. https://news.ycombinator.com/item?id=5634662 https://news.ycombinator.com/item?id=33251799 https://news.ycombinator.com/item?id=33270235 https://chatgpt.com/share/6aea25b4-e294-83e9-b925-431bb13b00...
Thanks for the mention. I ran across Declarative Drawing & I really want excuses to use this. But I don't remember Hazel from the video I watched in it!
Also this approach seems to introduce new complicated concepts like "composables" and even a DSL. Probably writing custom code for Drag-and-drop would be easier that figuring out how this works. I read the page and did not understand much. For example, this phrase is difficult to understand: "Rather than programming all that visual behavior directly, you simply say, for each element you want to make draggable, which future states of your application dragging it should lead to.". Not "simply" at all. Writing new code would be easier especially with LLM.
Here the goal can be a render state and it is the solvers problem to figure out what should be there.
So you define these things very atomically, but the behavior automagically finds a valid render state (usually through converting things into a standard planner or solver language, haven’t dived into this repo yet)
I fear this might lead to an exponential blowup.
(The examples shown are all tiny)
Edit: nvm, answered by the Background section of the paper
The Hazel programming env is dope. I can already imagine using it to explain refactorings to new programmers.
The twisted tree diagrams is also really neat. Is that a new idea as well? I got the beginnings of an idea where it might be a good diagram to display tangled code paths.
Very nice work